如何将数据从一个数字数组复制到另一个数组

将数据从数组 b 复制到数组 a 而不修改数组 a 的地址的最快方法是什么? 我需要这个，因为外部库(PyFFTW)使用一个指向我的数组的指针，这个指针不能更改。

例如:

a = numpy.empty(n, dtype=complex)
for i in xrange(a.size):
a[i] = b[i]

没有循环也可以做到吗？

113380 次浏览
小开
最佳答案

I believe

a = numpy.empty_like(b)
a[:] = b

will copy the values quickly. As Funsi mentions, recent versions of numpy also have the copyto function.

小开

you can easy use:

b = 1*a

this is the fastest way, but also have some problems. If you don't define directly the dtype of a and also doesn't check the dtype of b you can get into trouble. For example:

a = np.arange(10)        # dtype = int64
b = 1*a                  # dtype = int64


a = np.arange(10.)       # dtype = float64
b = 1*a                  # dtype = float64


a = np.arange(10)        # dtype = int64
b = 1. * a               # dtype = float64

I hope, I could make the point clear. Sometimes you will have a data type change with just one little operation.

小开

NumPy version 1.7 has the numpy.copyto function that does what you are looking for:

numpy.copyto(dst, src)

Copies values from one array to another, broadcasting as necessary.

See: https://docs.scipy.org/doc/numpy/reference/generated/numpy.copyto.html

小开 
a = numpy.array(b)

is even faster than the suggested solutions up to numpy v1.6 and makes a copy of the array as well. I could however not test it against copyto(a,b), since I don't have the most recent version of numpy.

小开

To answer your question, I played with some variants and profiled them.

Conclusion: to copy data from a numpy array to another use one of the built-in numpy functions numpy.array(src) or numpy.copyto(dst, src) wherever possible.

Update 2022-05: re-test with numpy v1.22 and CPython v3.9 showed that src.astype(...) is currently fastest almost consistently on my system. So, better run the provided code snipped yourself to get numbers for your specific setup.

(But always choose numpy.copyto(dst, src) if dst's memory is already allocated, to reuse the memory. See profiling at the end of the post.)

profiling setup

import timeit
import numpy as np
import pandas as pd
from IPython.display import display
    

def profile_this(methods, setup='', niter=10 ** 4, p_globals=None, **kwargs):
if p_globals is not None:
print('globals: {0}, tested {1:.0e} times'.format(p_globals, niter))
timings = np.array([timeit.timeit(method, setup=setup, number=niter,
globals=p_globals, **kwargs) for
method in methods])
ranking = np.argsort(timings)
timings = np.array(timings)[ranking]
methods = np.array(methods)[ranking]
speedups = np.amax(timings) / timings


# pd.set_option('html', False)
data = {'time (s)': timings,
'speedup': ['{:.2f}x'.format(s) if s != 1 else '' for s in speedups],
'methods': methods}
data_frame = pd.DataFrame(data, columns=['time (s)', 'speedup', 'methods'])


display(data_frame)
print()

profiling code

setup = '''import numpy as np; x = np.random.random(n)'''
methods = (
'''y = np.zeros(n, dtype=x.dtype); y[:] = x''',
'''y = np.zeros_like(x); y[:] = x''',
'''y = np.empty(n, dtype=x.dtype); y[:] = x''',
'''y = np.empty_like(x); y[:] = x''',
'''y = np.copy(x)''',
'''y = x.astype(x.dtype)''',
'''y = 1*x''',
'''y = np.empty_like(x); np.copyto(y, x)''',
'''y = np.empty_like(x); np.copyto(y, x, casting='no')''',
'''y = np.empty(n)\nfor i in range(x.size):\n\ty[i] = x[i]'''
)


for n, it in ((2, 6), (3, 6), (3.8, 6), (4, 6), (5, 5), (6, 4.5)):
profile_this(methods[:-1:] if n > 2 else methods, setup,
niter=int(10 ** it), p_globals={'n': int(10 ** n)})

results for Windows 7 on Intel i7 CPU, CPython v3.5.0, numpy v1.10.1.

globals: {'n': 100}, tested 1e+06 times


time (s) speedup                                            methods
0    0.386908  33.76x                                    y = np.array(x)
1    0.496475  26.31x                              y = x.astype(x.dtype)
2    0.567027  23.03x              y = np.empty_like(x); np.copyto(y, x)
3    0.666129  19.61x                     y = np.empty_like(x); y[:] = x
4    0.967086  13.51x                                            y = 1*x
5    1.067240  12.24x  y = np.empty_like(x); np.copyto(y, x, casting=...
6    1.235198  10.57x                                     y = np.copy(x)
7    1.624535   8.04x           y = np.zeros(n, dtype=x.dtype); y[:] = x
8    1.626120   8.03x           y = np.empty(n, dtype=x.dtype); y[:] = x
9    3.569372   3.66x                     y = np.zeros_like(x); y[:] = x
10  13.061154          y = np.empty(n)\nfor i in range(x.size):\n\ty[...




globals: {'n': 1000}, tested 1e+06 times


time (s) speedup                                            methods
0  0.666237   6.10x                              y = x.astype(x.dtype)
1  0.740594   5.49x              y = np.empty_like(x); np.copyto(y, x)
2  0.755246   5.39x                                    y = np.array(x)
3  1.043631   3.90x                     y = np.empty_like(x); y[:] = x
4  1.398793   2.91x                                            y = 1*x
5  1.434299   2.84x  y = np.empty_like(x); np.copyto(y, x, casting=...
6  1.544769   2.63x                                     y = np.copy(x)
7  1.873119   2.17x           y = np.empty(n, dtype=x.dtype); y[:] = x
8  2.355593   1.73x           y = np.zeros(n, dtype=x.dtype); y[:] = x
9  4.067133                             y = np.zeros_like(x); y[:] = x




globals: {'n': 6309}, tested 1e+06 times


time (s) speedup                                            methods
0  2.338428   3.05x                                    y = np.array(x)
1  2.466636   2.89x                              y = x.astype(x.dtype)
2  2.561535   2.78x              y = np.empty_like(x); np.copyto(y, x)
3  2.603601   2.74x                     y = np.empty_like(x); y[:] = x
4  3.005610   2.37x  y = np.empty_like(x); np.copyto(y, x, casting=...
5  3.215863   2.22x                                     y = np.copy(x)
6  3.249763   2.19x                                            y = 1*x
7  3.661599   1.95x           y = np.empty(n, dtype=x.dtype); y[:] = x
8  6.344077   1.12x           y = np.zeros(n, dtype=x.dtype); y[:] = x
9  7.133050                             y = np.zeros_like(x); y[:] = x




globals: {'n': 10000}, tested 1e+06 times


time (s) speedup                                            methods
0  3.421806   2.82x                                    y = np.array(x)
1  3.569501   2.71x                              y = x.astype(x.dtype)
2  3.618747   2.67x              y = np.empty_like(x); np.copyto(y, x)
3  3.708604   2.61x                     y = np.empty_like(x); y[:] = x
4  4.150505   2.33x  y = np.empty_like(x); np.copyto(y, x, casting=...
5  4.402126   2.19x                                     y = np.copy(x)
6  4.917966   1.96x           y = np.empty(n, dtype=x.dtype); y[:] = x
7  4.941269   1.96x                                            y = 1*x
8  8.925884   1.08x           y = np.zeros(n, dtype=x.dtype); y[:] = x
9  9.661437                             y = np.zeros_like(x); y[:] = x




globals: {'n': 100000}, tested 1e+05 times


time (s) speedup                                            methods
0   3.858588   2.63x                              y = x.astype(x.dtype)
1   3.873989   2.62x                                    y = np.array(x)
2   3.896584   2.60x              y = np.empty_like(x); np.copyto(y, x)
3   3.919729   2.58x  y = np.empty_like(x); np.copyto(y, x, casting=...
4   3.948563   2.57x                     y = np.empty_like(x); y[:] = x
5   4.000521   2.53x                                     y = np.copy(x)
6   4.087255   2.48x           y = np.empty(n, dtype=x.dtype); y[:] = x
7   4.803606   2.11x                                            y = 1*x
8   6.723291   1.51x                     y = np.zeros_like(x); y[:] = x
9  10.131983                   y = np.zeros(n, dtype=x.dtype); y[:] = x




globals: {'n': 1000000}, tested 3e+04 times


time (s) speedup                                            methods
0   85.625484   1.24x                     y = np.empty_like(x); y[:] = x
1   85.693316   1.24x              y = np.empty_like(x); np.copyto(y, x)
2   85.790064   1.24x  y = np.empty_like(x); np.copyto(y, x, casting=...
3   86.342230   1.23x           y = np.empty(n, dtype=x.dtype); y[:] = x
4   86.954862   1.22x           y = np.zeros(n, dtype=x.dtype); y[:] = x
5   89.503368   1.18x                                    y = np.array(x)
6   91.986177   1.15x                                            y = 1*x
7   95.216021   1.11x                                     y = np.copy(x)
8  100.524358   1.05x                              y = x.astype(x.dtype)
9  106.045746                             y = np.zeros_like(x); y[:] = x

Also, see results for a variant of the profiling where the destination's memory is already pre-allocated during value copying, since y = np.empty_like(x) is part of the setup:

globals: {'n': 100}, tested 1e+06 times


time (s) speedup                        methods
0  0.328492   2.33x                np.copyto(y, x)
1  0.384043   1.99x                y = np.array(x)
2  0.405529   1.89x                       y[:] = x
3  0.764625          np.copyto(y, x, casting='no')




globals: {'n': 1000}, tested 1e+06 times


time (s) speedup                        methods
0  0.453094   1.95x                np.copyto(y, x)
1  0.537594   1.64x                       y[:] = x
2  0.770695   1.15x                y = np.array(x)
3  0.884261          np.copyto(y, x, casting='no')




globals: {'n': 6309}, tested 1e+06 times


time (s) speedup                        methods
0  2.125426   1.20x                np.copyto(y, x)
1  2.182111   1.17x                       y[:] = x
2  2.364018   1.08x                y = np.array(x)
3  2.553323          np.copyto(y, x, casting='no')




globals: {'n': 10000}, tested 1e+06 times


time (s) speedup                        methods
0  3.196402   1.13x                np.copyto(y, x)
1  3.523396   1.02x                       y[:] = x
2  3.531007   1.02x                y = np.array(x)
3  3.597598          np.copyto(y, x, casting='no')




globals: {'n': 100000}, tested 1e+05 times


time (s) speedup                        methods
0  3.862123   1.01x                np.copyto(y, x)
1  3.863693   1.01x                y = np.array(x)
2  3.873194   1.01x                       y[:] = x
3  3.909018          np.copyto(y, x, casting='no')

小开

There are many different things you can do:

a=np.copy(b)
a=np.array(b) # Does exactly the same as np.copy
a[:]=b # a needs to be preallocated
a=b[np.arange(b.shape[0])]
a=copy.deepcopy(b)

Things that don't work

a=b
a=b[:] # This have given my code bugs
小开

Why not to use

a = 0 + b

I think it is similar to previous multiplication but might be simpler.


提交答案