Edit:Pylab has been deprecated please see the current accepted answer
Ok, It seems the answer is to start ipython notebook with --pylab=inline.
so ipython notebook --pylab=inline
This has it do what I saw earlier and what I wanted it to do.
Sorry about the vague original question.
Note that --pylab is deprecated and has been removed from newer builds of IPython, The recommended way to enable inline plotting in the IPython Notebook is now to run:
%matplotlib inline
import matplotlib.pyplot as plt
See this post from the ipython-dev mailing list for more details.