The efficiency provided from vim/emacs is mostly afforded by their heavy keyboard use. In these programs you can do most anything directly from the keyboard, rather than having to stop and use the mouse.
I'd say the actual vim/emacs editors are far superior to the Eclipse text editor in terms of the shortcuts they offer. However, I completely agree with you about refactoring.
Most people have to write scripts to do the sort of level of refactoring Eclipse is capable of. I think part of it is bragging rights or people just doing it the same way they always have.
Emacs can be a powerful IDE, but having gone from Emacs to Eclipse, I have to say I would never go back. Eclipse just offers so many features that you can't get within Emacs.
Mylyn and scoped views of the data and files I'm using, the debugging UI, CVS UI, are all built in and easy to get and use. I'll use the mouse a little to get'em.
I got started in IDEs like Eclipse, but switched to Vim about 2 years ago.
Reasons you may want to use a text-mode editor:
It can be used as an IDE for just about any language (you learn it once and use it for everything)
It can do all those fancy things like auto-completion, refactoring and many more complex operations, which you can extend by adding macros or plug-ins
It works just about everywhere (and can be used through an SSH shell)
You don't need a GB of ram to run it
If you really persevere, you will find that working in an editor like this will eventually be faster, and in fact becomes ingrained as a sort of 'muscle memory'. This means you can code without slowing down to think about the process.
I typically find myself writing volumes of code through vim and performing debugging tasks through my IDE. Familiarity with the code base is certainly a factor, as an IDE is a great tool for jumping around and learning unfamiliar source code.
I remember reading somewhere about a study which showed that people perceived keyboard shortcuts as more efficient than mousing, when in fact it wasn't always.
Another psychological effect is that we attach value to things which are expensive, i.e. since Emacs is harder to learn it must be better in some way.
I think those effects could explain a lot of the extreme affection some people have for Emacs/Vi.
However, in the case of Eclipse, I find it can be very slow and even crash occasionally, but that is not a case against IDEs in general.
I use both Eclipse, VS and Emacs regularly. I would use TextMate too, but I don't have a mac anymore. It depends on what I am doing, more specifically, what system best supports my language and tools.
I know people who spend considerably more time programming their editor, than they spend doing something useful. Some of them even admit themselves that they only do it for the challenge. Other people often claim that Emacs/Vi can do much more than IDEs, because they are scriptable. Well, most IDEs (including Eclipse) can also be scripted. In that sense almost all editors are equivalent (though, I admit, some editors are more easily scripted than others).
If you like IDEs, my advice is to keep using one. There is no One True Editor.
EDIT:
This seems to be the article Nick Bastin is referring to. I agree that it is far from a definitive source. However, I still think my point about perceived and real productivity not being the same thing still holds.
I've used vi for years to edit code in a variety of languages, and really love it. But I've found IDEs like Eclipse to be even nicer for Java development, and now I tend to work in Eclipse almost entirely. I drop out to use vi from time to time for a few specialized activities like bulk-inserting copyright notices, mainly those dealing with certain kinds of rote edits that aren't well automated in Eclipse. I also have my Windows .java file type mapped to vi for when I just want to look at a source file without waiting for Eclipse to open up.
Some of the attractive features in Eclipse are:
method name completion
error highlighting
pop-up javadoc comments
refactoring
I do find it a lot more efficient than vi in general, so you should try it out and see if it that holds true for you too.
It all depends on what you want/expect and what your usage model is.
If you're looking for a Java IDE, Eclipse is difficult to beat. It's written in Java, for Java, by Java folks.
If you're looking for a tool to edit files from the command line quickly, Emacs or vi both fit the bill.
If you're looking for a tool from which you never have to leave because it can do anything you want (send/read mail, manage projects, todo lists, compile, debug, etc. etc. etc.), then Emacs is more "efficient".
If you're looking for reasons to switch editors, figure out what you want. If you want a better Eclipse, vi and Emacs won't give you that, stick to Eclipse.
If you're looking for a small, nimble editor, vi will fit the bill.
If you're looking for the ultimately extensible editor, Emacs is the way.
Whichever tool you decide to go with, immerse yourself. Learn all of the ins and outs, extend it to meet your needs. Use it to its limits and become efficient in its use.
The argument "Eclipse for Java" is a different argument than "Eclipse for [something that isn't java]". Eclipse does rock for Java.
I mean, vi is like a screwdriver, or maybe a swiss army knife, and Eclipse is like a big CNC combo mill and asphalt spreader. You don't exactly compare them, you kind of just use both.
Also, are you working inside something giant, which you know little about, but which Eclipse understands? An example would be working on Eclipse itself. Here, Eclipse has perfect visibility, total language support, and you need the toast prompts and the documentation links.
But if you are typing in a 100-line Ruby program to convert an SQL database, Eclipse doesn't add much value, especially considering its baggage.
It's also critical to set up vi right, or you won't grok the appeal. Autoindent, showmatch, tab handling, and various other options should be set. You should have a easy way to generate a tags file. Google can find tag generators, or just write one from scratch, with a few lines of shellcode and sed(1).
I don't consider refactoring to be a criteria. That's not something you do once an hour or even once a day. Sure, fire up the big IDE when you need to refactor. Oh, and don't expect automated refactoring of anything except Java.
Finally, vi can actually do a lot of things that the IDE can't begin to do. The grouped regular expression global substitution is kind of a generalized refactor-anything engine. To appreciate the vi gestalt you need to learn the line (":") mode. Briefly, it's like having sed(1) inside your editor.
We've had this argument at work recently. My take was that one single feature I couldn't do without is Emacs's autocomplete. Eclipse's autocomplete is based on syntactical analysis - the code gets parsed, and as you type code you're offered choice of completion. Emacs' autocomplete is base on simple textual analysis. That means it works in plain text, in comments, in documentation - everywhere. I keep saying the Emacs' autocomplete is what IntelliSense wants to be when it grows up.
Update:
Eclipse does offer Alt-/ which is supposed to be similar to Emacs. Not sure how well it works, though.
Like some of the posts above, I started out with an IDE (Eclipse). From there I moved to Emacs, and then I moved back to a rich text-editor (TextMate).
For me, the efficiency was the ability to have an editor at the interface level. Allowing me to integrate other service I've built up (or others) into my pseudo-IDE environment.
The only place I prefer an IDE is for debugging. I set up my vim environment for debugging but is was so painful to use, so clunky, that I now just switch to my IDE (Netbeans) when I need to debug. vim is great for text editing, the IDE is great for more complex stuff (like debugging, and some project management related tasks).
First things first. VIM is more productive for programming than Eclipse. Your personal productity in VIM may be abysmal, but the potential cap of VIM is much higher. This is a fact.
VIM is a martial art. It feels unnatural when you first use it. And you can't even make it work. It takes years of practice to gradually become productive. You focus on mastering a little detail at first. Slowly all these bits you master add up until text is flowing effortlessly out of your finger tips onto the screen. Complicated edits that would make your co-worker sigh will jump from your hands before he can finish his exhale. There are few people who can use VIM. Fewer who can use it productivley. And you may never meet a master in your life time. But they are rumored to exist.
VIM is designed to keeep your hands on home-row. Moving your hand from the keyboard to the mouse is demoralizing. It's a gross motor movement. Moving your arm has a phsycological effect that hurts your motivation. Using VIM, someone could bolt your wrists onto the keyboard and you could still easily open up files, split windows, open tabs, build the project, search/replace, change fonts, change colors, etc. And all at lightning speed.
VIM is modal. That means you don't have to do complex key combinations where you hold down control+shift+Key. This hurts your hands in the long run. Instead you execute commands. There is no need for key combos to due to the modal nature.
We store data in our memory like computers do. Our memory can only hold a few values at a time. See how many distinct integers you can hold in your head before they start to slip away. We overcome this human limitation by writing stuff down. If data falls out of our memory we can easily look at what we wrote down to get it back. If your time is spent doing gross motor, physical things you are losing time that could have been spent on processing data in your brain. You want your mind to flow onto the screen without any effort at all. It may not sound like much but VIM's ability to effortlessly transfer what is in your mind to the screen is a BIG productivity boost. It's hard to put in words what I'm trying to say.
VIM supports code completion. Both textual and look-up based. It can pull text from mulitple files. Anything you desire can be had in VIM. Either make it yourself or use something someone else cooked up.
VIM supports goign to definitions with ctags. You can also find all references of an item. Again, anything you desire can be had in VIM.
The scripting of VIM is huge. You can download or create thousands of color schemes and change colors in an instant. Try to change fonts or colors in Microsoft Visual Studio and it will hang for 20 seconds while it loads data. It won't let you store color schemes and you must spend 30 minutes tweaking your colors and fonts every time you want a change of scenery. In VIM you can set line spacing to zero to fit more lines of code on the screen. I get over 80 lines. Visual studio uses 2 pixels of spacing for every line and you can't adjust it!!! Less lines = more scrolling = less productivity = forced to use small fonts for more lines = eye strain.
Split windows are opened in an instnat in VIM. It's usefull when you need to look at data in one section of the code that's far from the place you are typing (or in a different file). You don't have to spend time resizing windows, or worry about GUI windows overlapping each other and falling behind each other. Un-related code windows can be opened in tabs as to not take up screen space, but allow quick switching.
I would anytime go for emacs rather than eclipse. I also have to say that bare bone emacs, is not that great, but after some tweaking, you will never want to let it go. In particular I will tell you how helpful emacs was while writing my Master's thesis, that should make clear why Eclipse is inferior, just because it is less versatile.
I my master thesis I wrote in the following programming languages: C++, Python and R. Complementary, I had to write the thesis, for which I used LaTeX. Moreover, I had to write a bunch of shell scripts and cmake scripts. Guess what? Emacs has great support for all of it. Specially, it was a pleasure to work with AuCTeX to produce LaTeX documents. Then, Emacs provides the great ESS mode for working with R. Likewise, it provides facilities for python. Once I had my cmake scripts for building the C++ code, I only had to call compile from withing emacs and I was done. Eclipse cannot do this things altogether, therefore you will need to learn to use many different programs. Note taking? There is org-mode for that, and it is great!
And then, my program needed of a very powerful computer (not like any laptop). So, I could just do everything remotely from within emacs!!!! Using tramp, I found myself doing remote interactive evaluation of R code, remote compiling, executing and debuging C++ code, and everything within the same good local emacs window I had been using. In contrast, my friends who used a separate tool for everything were much slower in developing software that was meant to run in another computer.
Like this I have some other stories, but I think, this will give you a good idea on the things you can do with Emacs. All in all , I think choosing to use emacs (despite the learning curve) has been the most productive decision I have ever made.