Building workspace is about incremental build of any evolution detected in one of the opened projects in the currently used workspace.
You can also disable it through the menu "Project / Build automatically".
But I would recommend first to check:
if a Project Clean all / Build result in the same kind of long wait (after disabling this option)
if you have (this time with building automatically activated) some validation options you could disable to see if they have an influence on the global compilation time (Preferences / Validations, or Preferences / XML / ... if you have WTP installed)
if a fresh eclipse installation referencing the same workspace (see this eclipse.ini for more) results in the same issue (with building automatically activated)
Note that bug 329657 (open in 2011, in progress in 2014) is about interrupting a (too lengthy) build, instead of cancelling it:
There is an important difference between build interrupt and cancel.
When a build is cancelled, it typically handles this by discarding incremental build state and letting the next build be a full rebuild. This can be quite expensive in some projects.
As a user I think I would rather wait for the 5 second incremental build to finish rather than cancel and result in a 30 second rebuild afterwards.
The idea with interrupt is that a builder could more efficiently handle interrupt by saving its intermediate state and resuming on the next invocation.
In practice this is hard to implement so the most common boundary is when we check for interrupt before/after calling each builder in the chain.
For anyone running into a problem where build automatically is unchecked but the project is still building. Make sure your project isn't deployed to the server in the server tab and told to stay synchronous.