In 2008 I tried doing the same thing. I quickly noticed the following show stoppers:
Toolboxes are not as complete and not as well tested. Particularly the image processing toolbox that my work relied heavily upon (the big show stopper was that imtransform was not implemented).
The Octave debugger and profiler were primitive compared to Matlab's.
If you work with others, it may be very difficult to get them to change.
If you use third party toolboxes, you are on your own getting them to work.
Octave's plots are not publication quality.
But I have to say that I was generally impressed at how compatible Octave is with Matlab, if your use of Matlab is basic, you may get lucky. Finally this was in 2008, in two years things can change a lot.
It's interesting to see how the open source alternative works for statistics but not for numerical analysis. R (the octave of statistics) is nowadays much popular than the commercial S-plus (the matlab of statistics). The issues mentioned as reasons not to switch away from matlab found in the other answers were also applicable to R. But still everybody just started contributing and now R is the standard, with better graphics, better packages and no more vendor lock-in.
So you could prefer octave over matlab as well, if you can step over the prisoners dilemma.
Regarding octave: I was very impressed with the similarity of octave syntax. It didn't take me much time to transport my MATLAB scripts to octave. Meanwihile I have a particular problem on printing markers jointly with errorbar wich was fixed by Jarno Rajahalme at nabble and to change the xtick font size, which workaround I got in a question response at nabble. So it still have some bugs which with some effort can be overcome. If you experience some problems you may try nabble mailing forum: help-octave@octave.org. By the way my team cannot adapt (user friendly) to it such as they adapt to MATLAB, so we're still using MATLAB. Since MATLAB is built under gnuplot, another way to correct its bugs is editing the generated gnuplot file. The best IDE I found to it was QtOctave, that I made a short review in "Remember Blog".
Regarding R: according to a research made by SciViews, R's performance is superior to MATLAB and octave. I don't have much experience with R. I studied mclust package to wrote a wikibook chapter about EM Clustering in R. By the way, they seem to have a very active community. So you may find third party packages to proposals, which are not IMO so standardized. The best IDE I found was StatET plugin for eclipse, JGR (Java GUI for R) and emacs. Despite the time cost to learn a new programming language, if I would choose an open source platform to make my experiment graphics and some data mining analysis I would try R.
In my experience, core MATLAB is well ported to Octave, but the toolboxes have varying levels of compatibility, so your decision depends on what exactly you are trying to code.
Some things that Octave lacks, AFAIK, are the tight integration with .NET code and the gui builder, guide (though there are many other GUI builing tools that Octave can use).
Also, as others have pointed out, much of what you pay for with MATLAB is the slick interface and debugging/profiling tools. Experienced coders can probably manage with the alternatives, but newbies may struggle.
Note that Octave supports language constructs that aren't present in Matlab (e.g., auto-increment operators, do-until statements, etc.). This makes it sometimes annoying to port code developed (by someone who isn't familiar with the limitations of Matlab) on Octave to a Matlab environment.
For your use case, octave may be superior to MATLAB:
It has syntax that will allow you to
write code that is slightly closer to
C. i.e. +=, -=, default function
parameter values, double-quoted string
literals, etc...
Assuming your chips are slower than a
desktop processor, speed will likely
not be an issue.
Since it launches far faster than
matlab, it is more practical to
integrate into shell scripts for testing.
For prototyping, the plotting is more than adequate; people are just used to MATLAB's style.
The relative lack of toolboxes isn't
a big deal since they wouldn't be
available on your target platform
anyway.
I use both, and whenever I switch, I miss features from the other.
Octave has several syntactic improvements on matlab, for example you can say endifendfor and endfunction instead of just end, which make debugging much easier.
Octave also allows you to dynamically generate functions, and have multiple functions defined in scripts and function file. Which is way nicer than matlab's one-file-one-function approach.
Finally, octave has parcellfun and pararrayfun which are very powerful parallel processing tools which matlab completely lacks. There is a parfor in matlab, but it's not the best way of doing it in my opinion.
Cons for octave are that they are slightly behind on toolboxes, though if you look you can find things similar. fsolve and lsode seem a little slower, but more robust, in octave for some reason. Also a big bummer for some people tends to be the lack of symlink and the DAQ toolbox, but that stuff is going to be proprietary anyway.
Python/Numpy is definitely worth a whirl: it's more powerful but their syntax is aimed at more complex pieces of code.
I have successfully ported some linear regression and quadratic programming applications to Octave.
The linear regression (backslash operator) worked without any adjustment. In case of quadratic programming I had to switch from fmincon() to sqp(), giving similar results.
Still, the toolboxes and GUI in Octave are, indeed, less mature (I spent so much time on basic stuff), although it has been rapidly making progress over the past two years.