numpy-like package for node

During my years on Python development, I've always been amazed at how much much much faster things become if you manage to rewrite that code that loops though your ndarray and does something, with numpy functions that work on the whole array at once. More recently I'm switching more and more to node, and I'm looking for something similar. So far I have turned up some things, none of which look promising:

  • scikit-node, runs scikit-learn in python, and interfaces with node. I haven't tried it, but I don't expect it gives me the cutting edge speed that I would like.
  • There are some rather old, and newer, javascript matrix libraries (sylvester, gl-matrix, ...). In addition to not being sure they work well with matrices larger than 4x4 (which is most useful in 3D rendering), they seem to be native javascript (and some, not sure these, use webGL acceleration). Great on the browser, not so on node.

As far as I know, npms can be written in C++, so I'm wondering why there are no numpy-like libraries for node. Is there just not enough interest in node yet from the community that needs that kind of power? Is there a hope that ES6 features (list comprehensions) will allow javascript compilers to automatically vectorise native JS code to C++ speeds? Am I possibly missing something else?

Edit, in response to close-votes: Note, I'm not asking for "what is the best package to do xyz". I'm just wondering if there is a technical reason there is no package to do this on node, a social reason, or no reason at all and there is just a package I missed. Maybe to avoid too many opinionated criticism, I want to know: I have about 10000 matrices that are 100 x 100 each. What's the best (* correction, a reasonable fast) way to add them together?

Edit2 After some more digging, it turned out I was googling for the wrong thing. Google for "node.js scientific computing" and there are links to some very interesting notes:

Basically as far as I understand now, no-one has bothered so far. Also, since there are some major omissions in the js TypedArrays (such as 64bit ints), it might be hard to add good support by just using NPMs, and not hacking the engine itself --- something that would defeat the purpose. Then again, I didn't further research this last statement.

50655 次浏览
小开

The majority of node work seems to be in the web "full stack" universe, with a lot less work being done in areas where fast numeric processing is an advantage.

In the areas where fast numeric processing is an advantage, Python, R, etc probably have a dominant mindshare.

Combine those two facts, and you end up with not a lot of people putting effort into node numeric processing libraries.

小开

I've not tried this, but I found node-lapack. Since Numpy gets most of it's speed from using blas/lapack to do everything, this should help. From the readme it looks like it has an array object too, which is essential to not convert between JS and lapack on every operation.

Here's a part of their demo:

var lapack = require('lapack');


var result = lapack.sgeqrf([
[1, 2, 3],
[3, 4, 5],
[5, 6, 7]
]);


console.log(result.R);
console.log(result.tau);


result = sgesvd('A', 'A', [
[1, 2, 3],
[3, 4, 5],
[5, 6, 7]
]);


console.log(result.U);
console.log(result.S);
console.log(result.VT);


result = lapack.sgetrf([
[1, 2, 3],
[3, 4, 5],
[5, 6, 7]
]);


// see the readme for more

It seems to be a pretty direct interface to lapack using the same names, so in that regard it's not as convenient as Numpy, but at least it takes care of array dimensions and stuff and should be about as fast (since most of the work is being done by Lapack in either case).

However, this won't work in a browser, which means everywhere where this is available, Python is probably also available. Personally I'd stick with Python, which is much more dominant for numerical stuff, unless there's some specific Node functionality Python is missing...

小开

No, there are no technical reasons why a numpy-like package does not exist for Node.js and, more generally, JavaScript.

There are two main obstacles preventing Node.js and JavaScript from achieving more mind share in the data science and numeric computing communities.

The first obstacle is community. While the JavaScript community is huge, the subset of people within that community doing interesting things in numeric computing is small. Hence, if you want to do numeric computing in JavaScript and Node.js, finding resources to help you along the way can be hard, and it may feel like a lonely endeavor.

Next, the absence of comparable libraries (chicken and egg: libraries are needed to attract library authors and authors are needed to write good libraries). There are no technical reasons why libraries cannot be written in JavaScript or leverage Node.js (e.g., via native add-ons). I know, as I have written many numeric computing libraries in JavaScript. So while numeric computing is possible in JavaScript, the problem stems from an inability to attract developers having sufficient expertise and capable of putting in the time and effort needed to write high quality numeric computing implementations.

Regarding the specific language features mentioned in the OP:

  • ES6/ES2015: none of the recent language additions help or hinder development of numeric computing libraries in JavaScript. Potential additions like list comprehensions will not be game changers either. The one change to the web platform which will make a difference is WebAssembly. With WebAssembly, compiling C/C++/Fortran libraries to run in web browsers will be made easier. At the time of this answer, WebAssembly looks to be the means for bringing SIMD to the web, potentially allowing some speed-ups, although the focus seems to be on short SIMD, rather than long. But even with WebAssembly, porting numeric computing libraries to the web will not be as simple as hitting the compile button. Numeric computing code bases will need to massaged to become amenable for use on the web, and, even then, higher level APIs will likely need to be written to mask some of lower level features, such as manually managing the heap.
  • Native add-ons: yes, node modules can be written as native add-ons, allowing C/C++/Fortran code to be used within a Node.js application. Individuals have written libraries to this end; for example, see stdlib. If done well, Node.js can perform numeric computations at speeds comparable to directly using native implementations.
  • Typed arrays: as they are now, they are suitable for numeric computation. Similar to C, you can create pooled buffers, which allow for efficient memory reuse and better performance. Furthermore, similar to languages like R, Python, and Julia, you can leverage typed arrays to create ndarray (aka strided array) interfaces. While U/Int64 integer arrays are not currently available at the time of this answer, (a) their absence is not a show stopper and (b) proposals are advancing at the specification level to add U/Int64 integer arrays to JavaScript. Ditto for complex numbers with structured types.

My personal belief is that some form of numeric computing is inevitable in JavaScript and Node.js. The advantages (ubiquity, distribution, performance) and potential applications (edge computing, integrating machine learning, data visualization) are too strong of evolutionary forces not to support data science applications, at least at a basic level.

disclosure: I and others are currently working on a project (https://github.com/stdlib-js/stdlib) which aims to provide numeric computing facilities in JavaScript and Node.js.

小开
最佳答案

Here is Google's TensorFlow.js (previously https://deeplearnjs.org), which does exactly that, and has built in capacities to train deep neural networks on GPUs using WebGL. You can also port TensorFlow models to it.

Don't be fooled into thinking this is only for deep learning. It is a fully fledged numerical computing platform with built-in GPU acceleration. It follows the eager "execute as you go" model, like NumPy (and Tensorflow Eager, and PyTorch, and others), not the "define then run" model like Tensorflow. As such, it will feel natural to use to anyone who has used NumPy before.

Here is the very informative Github repo:

https://github.com/tensorflow/tfjs-core (the old link https://github.com/PAIR-code/deeplearnjs now redirects there)

小开

In the same spirit of @Julius's answer about deeplearn.js, tensorflow.js is the continuation of the same project. To play around with the tensorflow module in the REPL, I installed it globally (FYI - it is usually advised not to do this) using this:

$ npm install --global @tensorflow/tfjs

Then, I ran $ node to start the node REPL.

This may differ for you (especially if you decided to install tensorflow locally), but I entered this to reference the tensorflow module:

var tf = require('/usr/local/lib/node_modules/@tensorflow/tfjs')

To create a rank 1 tensor (equivalent to a 1-D array in numpy), try:

var x = tf.tensor( [-3,4] )

And square it with:

x.square().print()

You should get [9,16] for your output. See https://js.tensorflow.org for more details.

I would say that tensorflow.js is not only a JS replacement for numpy, but also for sklearn, keras, and of course, tensorflow.

小开

numjs is a numpy like library in nodejs.

小开

I started https://www.npmjs.com/package/@nexys/math-ts a very simple and light numerical typescript library/package

Github: https://github.com/Nexysweb/math-ts

小开

scijs's ndarray is also good. link

var mat = ndarray(new Float64Array([1, 0, 0, 1]), [2,2])
 

//Now:
//
// mat = 1 0
//       0 1
//

提交答案