JUnit vs TestNG

I can't comment on TestNG b/c I haven't used it. But I would recommend unitils, a great wrapper for JUnit/TestNG/DBUnit/EasyMock, regardless of which route you take. (It supports all the flavors mentioned above)

Me and my colleague had the same issue using Azure DevOps. His branch was behind, and all we had to do was:

git pull origin master

I've used both, but I have to agree with Justin Standard that you shouldn't really consider rewriting your existing tests to any new format. Regardless of the decision, it is pretty trivial to run both. TestNG strives to be much more configurable than JUnit, but in the end they both work equally well.

from his local branch. This fixed it.

TestNG has a neat feature where you can mark tests as a particular group, and then easily run all tests of a specific group, or exclude tests of a particular group. Thus you can mark tests that run slowly as in the "slow" group and then ignore them when you want quick results. A suggestion from their documentation is to mark some subset as "checkin" tests which should be run whenever you check new files in. I never saw such a feature in JUnit, but then again, if you don't have it, you don't REALLY miss it.

For all its claims of high configuration, I did run into a corner case the a couple weeks ago where I couldn't do what I wanted to do... I wish I could remember what it is, but I wanted to bring it up so you know that it's not perfect.

The biggest advantage TestNG has is annotations... which JUnit added in version 4 anyways.

Take this warning seriously. This happens either

Also, JUnit is pretty much the de-facto standard in the Java world. There's no decent tool that doesn't support it from the box, you can find a lot of help on the web and they added a lot of new features in the past year which shows it's alive.

We decided to stick with JUnit and never looked back.

• When your source branch doesn't have the latest destination branch and you have created the pull request.

• Or you have merged the a branch which is not the origin of your current branch.

  

For the second reason: When you have pushed a branch to your current branch which is not the origin of your current branch.

For the first reason: Take your branch to local, merge the origin of the destination branch to your branch, commit and push. Check the PR now. It must be resolved now. If not, then you must check the second reason.

• The best solution, if feasible in your case, is to identify the branch which was not the origin of your branch and revert the commit and push it.

For the second reason: When you have pushed a branch to your current branch which is not the origin of your current branch.

• If not, then create a new branch from the origin of the destination, cherry pick all the commits which are relevant and commit push. Create a new PR from this new branch.
• The best solution, if feasible in your case, is to identify the branch which was not the origin of your branch and revert the commit and push it.

Choose the option based on your feasibility.

Whilst on the surface one might not think all of TestNGs features above might not be needed, once you start to understand the flexibility bring to your tests, you'll wonder how you coped with JUnit.

(disclaimer - I've not used JUnit 4.x at all, so am unable to really comment on advances or new features there).

NOTE: I've updated this to remove the wrong bits from my last answer. D'oh! (thanks puetzk and j_random_hacker for pointing it out)

While it's correct that one shouldn't generally add stuff to the std:: namespace, adding template specializations for user-defined types is specifically allowed. Overloading the functions is not. This is a subtle difference :-)

17.4.3.1/1 It is undefined for a C++ program to add declarations or definitions

There is a thread on comp.lang.c++.moderated with a long dicussion of the topic. Most of it is about partial specialization, though (which there's currently no good way to do).

Also one more advantage of TestNG is supporting of parallel testing. In our era of multicores it's important, i think.

I also used both frameworks. But i using hamcrest for assertations. Hamcrest allows you easily write your own assert method. So instead of

assertEquals(operation.getStatus(), Operation.Status.Active);

You can write

assertThat(operation, isActive());

That gives you opportunity to use higher level of abstraction in your tests. And this makes your tests more robust.

I wanted to share the one I encountered today. I found built-in Parameterized runner is quite crude in Junit4 as compare to TestNG (I know each framework has its strengths but still). The Junit4 annotation @parameters is restricted to one set of parameters. I encountered this problem while testing the valid and invalid behavior for functionality in same test class. So the first public, static annotated method that it finds will be used, but it may find them in any order. This causes us to write different classes unnecessarily. However TestNG provides clean way to provide different kind of data providers for each and every method. So we can test the same unit of code with valid and invalid way in same test class putting the valid/invalid data separately. I will go with TestNG.

Cheers to all the above. Some other things I've personally found I like more in TestNG are:

A specialization of std::swap would look like:

namespace std
{
template<>
void swap(myspace::mytype& a, myspace::mytype& b) { ... }
}

1. The @BeforeClass for TestNG takes place after class creation, so you aren't constrained by only being able to call static methods of your class in it.

Without the template<> bit it would be an overload, which is undefined, rather than a specialization, which is permitted. @Wilka's suggest approach of changing the default namespace may work with user code (due to Koenig lookup preferring the namespace-less version) but it's not guaranteed to, and in fact isn't really supposed to (the STL implementation ought to use the fully-qualified std::swap).

2. Parallel and parameterized tests, maybe I just don't have enough of a life... but I just get a kick writing one set of Selenium tests, accepting a driver name as a parameter. Then defining 3 parallel test groups, 1 each for the IE, FF and Chrome drivers, and watching the race! I originally did 4, but way too many of the pages I've worked on break the HtmlUnit driver for one reason or another.

Yeah, probably need to find that life. ;)

Attention Mozza314

Here is a simulation of the effects of a generic std::algorithm calling std::swap, and having the user provide their swap in namespace std. As this is an experiment, this simulation uses namespace exp instead of namespace std.

// simulate <algorithm>


#include <cstdio>


namespace exp
{


template <class T>
void
swap(T& x, T& y)
{
printf("generic exp::swap\n");
T tmp = x;
x = y;
y = tmp;
}


template <class T>
void algorithm(T* begin, T* end)
{
if (end-begin >= 2)
exp::swap(begin[0], begin[1]);
}


}


// simulate user code which includes <algorithm>


struct A
{
};


namespace exp
{
void swap(A&, A&)
{
printf("exp::swap(A, A)\n");
}


}


// exercise simulation


int main()
{
A a[2];
exp::algorithm(a, a+2);
}

Now for what I believe is a separate issue, how to write well structured, readable and maintainable tests. Most of this I am sure you know, but things like Factory Pattern, Command Pattern and PageObjects (if your testing websites) are vital, it is very important to have a layer of abstraction between what your testing (SUT) and what the actual test is (assertions of business logic). In order to have much nicer assertions, you can use Hamcrest. Make use of javas inheritance/interfaces to reduce repetition and enforce commonality.

For me this prints out:

generic exp::swap

Almost forgot, also use the Test Data Builder Pattern, this coupled with TestNG's dataprovider annotation is very useful.

If your compiler prints out something different then it is not correctly implementing "two-phase lookup" for templates.

// simulate <algorithm>


#include <cstdio>


namespace exp
{


template <class T>
void
swap(T& x, T& y)
{
printf("generic exp::swap\n");
T tmp = x;
x = y;
y = tmp;
}


template <class T>
void algorithm(T* begin, T* end)
{
if (end-begin >= 2)
swap(begin[0], begin[1]);
}


}


// simulate user code which includes <algorithm>


struct A
{
};


void swap(A&, A&)
{
printf("swap(A, A)\n");
}


// exercise simulation


int main()
{
A a[2];
exp::algorithm(a, a+2);
}

If your compiler is conforming (to any of C++98/03/11), then it will give the same output I show. And in that case exactly what you fear will happen, does happen. And putting your swap into namespace std (exp) did not stop it from happening.

Output is:

swap(A, A)

Update

Dave and I are both committee members and have been working this area of the standard for a decade (and not always in agreement with each other). But this issue has been settled for a long time, and we both agree on how it has been settled. Disregard Dave's expert opinion/answer in this area at your own peril.

An observation has been made that:

namespace exp
{
template <>
void swap(A&, A&)
{
printf("exp::swap(A, A)\n");
}


}

works! So why not use that?

This issue came to light after C++98 was published. Starting about 2001 Dave and I began to work this area. And this is the modern solution:

// simulate <algorithm>


#include <cstdio>


namespace exp
{


template <class T>
void
swap(T& x, T& y)
{
printf("generic exp::swap\n");
T tmp = x;
x = y;
y = tmp;
}


template <class T>
void algorithm(T* begin, T* end)
{
if (end-begin >= 2)
swap(begin[0], begin[1]);
}


}


// simulate user code which includes <algorithm>


struct A
{
};


void swap(A&, A&)
{
printf("swap(A, A)\n");
}


// exercise simulation


int main()
{
A a[2];
exp::algorithm(a, a+2);
}

Consider the case that your A is a class template:

// simulate user code which includes <algorithm>


template <class T>
struct A
{
};


namespace exp
{


template <class T>
void swap(A<T>&, A<T>&)
{
printf("exp::swap(A, A)\n");
}


}


// exercise simulation


int main()
{
A<int> a[2];
exp::algorithm(a, a+2);
}

Output is:

swap(A, A)

Now it doesn't work again. :-(

Update

So you could put swap in namespace std and have it work. But you'll need to remember to put swap in A's namespace for the case when you have a template: A<T>. And since both cases will work if you put swap in A's namespace, it is just easier to remember (and to teach others) to just do it that one way.

An observation has been made that:

namespace exp
{
template <>
void swap(A&, A&)
{
printf("exp::swap(A, A)\n");
}


}

My opinion about what makes TestNG truly far more powerful:

1.  JUnit still requires the before/after class methods to be static, which limits
what you can do prior to the running of tests, TestNG never has this issue.


2.  TestNG @Configuration methods can all take an optional argument to their
annotated methods in the form of a ITestResult, XmlTest, Method, or
ITestContext.  This allows you to pass things around that JUnit wouldn't
provide you.  JUnit only does this in listeners and it is limited in use.


3.  TestNG comes with some pre-made report generation classes that you can copy
and edit and make into your own beautiful test output with very little
effort. Just copy the report class into your project and add a listener
to run it.  Also, ReportNG is available.


4.  TestNG has a handful of nice listeners that you can hook onto so you can do
additional AOP style magic at certain phases during testing.

You can find more detailed comparison here.

Verging on religious but I would say that you're painting an overly grim picture of the state of modern OOP. I would argue that it actually has reduced costs, made large software projects manageable, and so forth. That doesn't mean it's solved the fundamental problem of software messiness, and it doesn't mean the average developer is an OOP expert. But the modularization of function into object-components has certainly reduced the amount of spaghetti code out there in the world.

A good difference is given in this tutorial side by side: TestNG Vs JUnit: What's the Difference?