edited body

Source Link

edited Jun 14, 2024 at 10:39

71.9k
15
100
126

reshaped = data.reshape(-1, m, n)
chunks = [reshaped[:, i, :] for i in range(m)]

For 1D:

reshaped = data.reshape(-1, m, n)
chunks = [reshaped[:, i, :].ravel() for i in range(nm)]

reshaped = data.reshape(-1, n, m)
chunks = [reshaped[:, i, :] for i in range(n)]

reshaped = data.reshape(-1, m, n)
chunks = [reshaped[:, i, :] for i in range(m)]

For 1D:

reshaped = data.reshape(-1, m, n)
chunks = [reshaped[:, i, :].ravel() for i in range(m)]

added 1099 characters in body

Source Link

edited Jun 14, 2024 at 10:21

U13-Forward

71.9k
15
100
126

TL;DR, for fastest solution, go to Solution #4#1.

Solution 1: Reshape and chunking

I use reshape to change the shape of the array, to chunks of 8 using (-1, 8).

If you don't understand what -1 does, check out the post here.

As mentioned in the comments my @BallpointBen:

Solution 1: Fastest one + Generalized:

... When reshaping an array, the new shape must contain the same number of elements as the old shape, meaning the products of the two shapes' dimensions must be equal. When using a -1, the dimension corresponding to the -1 will be the product of the dimensions of the original array divided by the product of the dimensions given to reshape so as to maintain the same number of elements.

We reshape the matrix to already done chunks, and then we use a list comprehension to slice the arrays and accomplish the task.

I then use numpy slicing to properly slice the matrix by chunks of 4reshaped[:, i, :] to filter out the specific chunk from the 3D matrix.

reshaped = data.reshape(-1, 8n, m)
achunks = reshaped[[reshaped[:, i, :4]
b] =for reshaped[:,i 4:in range(n)]

Above is the best and cleanest answer.!

Solution 4: Fastest one + Generalized:

We reshape the matrix to already done chunks, and then we use a list comprehension to slice the arrays and accomplish the task.Solution 4: Reshape and chunking

I use reshaped[:, ireshape to change the shape of the array, to chunks of 8 using (-1, :]8) to filter.

If you don't understand what -1 does, check out the specific chunk from the 3D matrixpost here.

As mentioned in the comments my @BallpointBen:

... When reshaping an array, the new shape must contain the same number of elements as the old shape, meaning the products of the two shapes' dimensions must be equal. When using a -1, the dimension corresponding to the -1 will be the product of the dimensions of the original array divided by the product of the dimensions given to reshape so as to maintain the same number of elements.

I then use numpy slicing to properly slice the matrix by chunks of 4.

reshaped = data.reshape(-1, n, m8)
chunksa = [reshaped[reshaped[:, i, :] for4]
b i= inreshaped[:, range(n)4:]

import timeit

def U13_1():
    # Generalized
    data = np.arange(1, 499875841)
    reshaped = data.reshape(-1, 8n, m)
    achunks = reshaped[[reshaped[:, :4]
  i, :] bfor =i reshaped[:,in 4:range(n)]

def U13_2():
    data = np.arange(1, 499875841)
    chunks = np.array_split(data, (data.shape[0] + 3) // 4)
    a, b = chunks[::2], chunks[1::2]

def U13_3():
    data = np.arange(1, 499875841)
    chunks = data.reshape((data.shape[0] // 4, 4))
    a, b = chunks[::2], chunks[1::2]

def U13_4():
    # Generalized
    data = np.arange(1, 499875841)
    reshaped = data.reshape(-1, n, m8)
    chunksa = [reshaped[reshaped[:, i, :]4]
 for i in range(n)b = reshaped[:, 4:]

def U13_5():
    data = np.arange(1, 499875841)
    chunks = [data[i:i+4] for i in range(len(data)) if (i % 4) == 0]
    a, b = chunks[::2], chunks[1::2]

def U13_6():
    data = np.arange(1, 499875841)
    chunks = [[data[i:i+4], data[i+4:i+8]] for i in range(len(data)) if (i % 8) == 0]
    a, b = zip(*chunks)

def hpaulj():
    # Generalized
    data = np.arange(1, 499875841)
    a,b,c = arr.reshape(-1, M, N).transpose(1, 0, 2).reshape(-1, arr.size//M) 

def mozway():
    # Generalized
    data = np.arange(1, 499875841)
    out = np.split(np.argsort(np.arange(len(data))%(4*2)//4, kind='stable'), 2)

def pauls():
    data = np.arange(1, 499875841)
    i = np.arange(1, len(data)+1) % 8 
    m = (i >= 1) & (i <= 4) 
    a = data[m]
    b = data[~m]

U13_1 = timeit.timeit('U13_1()', 'from __main__ import U13_1', number=1)
print('U13_1 Speed:', U13_1)
U13_2 = timeit.timeit('U13_2()', 'from __main__ import U13_2', number=1)
print('U13_2 Speed:', U13_2)
U13_3 = timeit.timeit('U13_3()', 'from __main__ import U13_3', number=1)
print('U13_3 Speed:', U13_3)
U13_4 = timeit.timeit('U13_4()', 'from __main__ import U13_4', number=1)
print('U13_4 Speed:', U13_4)
U13_5 = timeit.timeit('U13_5()', 'from __main__ import U13_5', number=1)
print('U13_5 Speed:', U13_5)
hpaulj = timeit.timeit('hpaulj()', 'from __main__ import hpaulj', number=1)
print('U13_6 Speed:', U13_6)
hpaulj = timeit.timeit('hpaulj()', 'from __main__ import hpaulj', number=1)
print('hpaulj Speed:', hpaulj)
mozway = timeit.timeit('mozway()', 'from __main__ import mozway', number=1)
print('mozway Speed:', mozway)
pauls = timeit.timeit('pauls()', 'from __main__ import pauls', number=1)
print('pauls Speed:', pauls)

import matplotlib.pyplot as plt
df = pd.DataFrame({'U13_1': U13_1, 'U13_2': U13_2, 'U13_3': U13_3, 'U13_4': U13_4, 'U13_5': U13_5, 'U13_6': U13_6, 'hpaulj': hpaulj, 'mozway': mozway, 'pauls': pauls}, index=[0])

plt.plot(df.columns, df.iloc[0])
plt.show()

U13_1 Speed: 0.690001300000403745211420000032376
U13_2 Speed: 378.43887300000006
U13_3 Speed: 0.6713313000000198
U13_4 Speed: 0.452114200000323766900013000004037
U13_5 Speed: 298.6191995999998
U13_6 Speed: 601.4342458000001
hpaulj Speed: 1.2215797999997449
mozway Speed: 21.53476950000004
pauls Speed: 4.3681056999994325

My solution #4#1 Generalized
My solution #3 Not Generalized
My solution #1#4 Not Generalized

My solution #4#1 Generalized
@hpaulj's solution Generalized
@mozway's solution Generalized

TL;DR, for fastest solution, go to Solution #4.

Solution 1: Reshape and chunking

I use reshape to change the shape of the array, to chunks of 8 using (-1, 8).

If you don't understand what -1 does, check out the post here.

As mentioned in the comments my @BallpointBen:

... When reshaping an array, the new shape must contain the same number of elements as the old shape, meaning the products of the two shapes' dimensions must be equal. When using a -1, the dimension corresponding to the -1 will be the product of the dimensions of the original array divided by the product of the dimensions given to reshape so as to maintain the same number of elements.

I then use numpy slicing to properly slice the matrix by chunks of 4.

reshaped = data.reshape(-1, 8)
a = reshaped[:, :4]
b = reshaped[:, 4:]

Above is the cleanest answer.

Solution 4: Fastest one + Generalized:

We reshape the matrix to already done chunks, and then we use a list comprehension to slice the arrays and accomplish the task.

I use reshaped[:, i, :] to filter out the specific chunk from the 3D matrix.

reshaped = data.reshape(-1, n, m)
chunks = [reshaped[:, i, :] for i in range(n)]

import timeit

def U13_1():
    data = np.arange(1, 499875841)
    reshaped = data.reshape(-1, 8)
    a = reshaped[:, :4]
    b = reshaped[:, 4:]

def U13_2():
    data = np.arange(1, 499875841)
    chunks = np.array_split(data, (data.shape[0] + 3) // 4)
    a, b = chunks[::2], chunks[1::2]

def U13_3():
    data = np.arange(1, 499875841)
    chunks = data.reshape((data.shape[0] // 4, 4))
    a, b = chunks[::2], chunks[1::2]

def U13_4():
    # Generalized
    data = np.arange(1, 499875841)
    reshaped = data.reshape(-1, n, m)
    chunks = [reshaped[:, i, :] for i in range(n)]

def U13_5():
    data = np.arange(1, 499875841)
    chunks = [data[i:i+4] for i in range(len(data)) if (i % 4) == 0]
    a, b = chunks[::2], chunks[1::2]

def U13_6():
    data = np.arange(1, 499875841)
    chunks = [[data[i:i+4], data[i+4:i+8]] for i in range(len(data)) if (i % 8) == 0]
    a, b = zip(*chunks)

def hpaulj():
    # Generalized
    data = np.arange(1, 499875841)
    a,b,c = arr.reshape(-1, M, N).transpose(1, 0, 2).reshape(-1, arr.size//M) 

def mozway():
    # Generalized
    data = np.arange(1, 499875841)
    out = np.split(np.argsort(np.arange(len(data))%(4*2)//4, kind='stable'), 2)

def pauls():
    data = np.arange(1, 499875841)
    i = np.arange(1, len(data)+1) % 8 
    m = (i >= 1) & (i <= 4) 
    a = data[m]
    b = data[~m]

U13_1 = timeit.timeit('U13_1()', 'from __main__ import U13_1', number=1)
print('U13_1 Speed:', U13_1)
U13_2 = timeit.timeit('U13_2()', 'from __main__ import U13_2', number=1)
print('U13_2 Speed:', U13_2)
U13_3 = timeit.timeit('U13_3()', 'from __main__ import U13_3', number=1)
print('U13_3 Speed:', U13_3)
U13_4 = timeit.timeit('U13_4()', 'from __main__ import U13_4', number=1)
print('U13_4 Speed:', U13_4)
U13_5 = timeit.timeit('U13_5()', 'from __main__ import U13_5', number=1)
print('U13_5 Speed:', U13_5)
hpaulj = timeit.timeit('hpaulj()', 'from __main__ import hpaulj', number=1)
print('U13_6 Speed:', U13_6)
hpaulj = timeit.timeit('hpaulj()', 'from __main__ import hpaulj', number=1)
print('hpaulj Speed:', hpaulj)
mozway = timeit.timeit('mozway()', 'from __main__ import mozway', number=1)
print('mozway Speed:', mozway)
pauls = timeit.timeit('pauls()', 'from __main__ import pauls', number=1)
print('pauls Speed:', pauls)

import matplotlib.pyplot as plt
df = pd.DataFrame({'U13_1': U13_1, 'U13_2': U13_2, 'U13_3': U13_3, 'U13_4': U13_4, 'U13_5': U13_5, 'U13_6': U13_6, 'hpaulj': hpaulj, 'mozway': mozway, 'pauls': pauls}, index=[0])

plt.plot(df.columns, df.iloc[0])
plt.show()

U13_1 Speed: 0.6900013000004037
U13_2 Speed: 378.43887300000006
U13_3 Speed: 0.6713313000000198
U13_4 Speed: 0.45211420000032376
U13_5 Speed: 298.6191995999998
U13_6 Speed: 601.4342458000001
hpaulj Speed: 1.2215797999997449
mozway Speed: 21.53476950000004
pauls Speed: 4.3681056999994325

My solution #4 Generalized
My solution #3 Not Generalized
My solution #1 Not Generalized

My solution #4 Generalized
@hpaulj's solution Generalized
@mozway's solution Generalized

TL;DR, for fastest solution, go to Solution #1.

Solution 1: Fastest one + Generalized:

We reshape the matrix to already done chunks, and then we use a list comprehension to slice the arrays and accomplish the task.

I use reshaped[:, i, :] to filter out the specific chunk from the 3D matrix.

reshaped = data.reshape(-1, n, m)
chunks = [reshaped[:, i, :] for i in range(n)]

Above is the best and cleanest answer!

Solution 4: Reshape and chunking

I use reshape to change the shape of the array, to chunks of 8 using (-1, 8).

If you don't understand what -1 does, check out the post here.

As mentioned in the comments my @BallpointBen:

... When reshaping an array, the new shape must contain the same number of elements as the old shape, meaning the products of the two shapes' dimensions must be equal. When using a -1, the dimension corresponding to the -1 will be the product of the dimensions of the original array divided by the product of the dimensions given to reshape so as to maintain the same number of elements.

I then use numpy slicing to properly slice the matrix by chunks of 4.

reshaped = data.reshape(-1, 8)
a = reshaped[:, :4]
b = reshaped[:, 4:]

import timeit

def U13_1():
    # Generalized
    data = np.arange(1, 499875841)
    reshaped = data.reshape(-1, n, m)
    chunks = [reshaped[:, i, :] for i in range(n)]

def U13_2():
    data = np.arange(1, 499875841)
    chunks = np.array_split(data, (data.shape[0] + 3) // 4)
    a, b = chunks[::2], chunks[1::2]

def U13_3():
    data = np.arange(1, 499875841)
    chunks = data.reshape((data.shape[0] // 4, 4))
    a, b = chunks[::2], chunks[1::2]

def U13_4():
    data = np.arange(1, 499875841)
    reshaped = data.reshape(-1, 8)
    a = reshaped[:, :4]
    b = reshaped[:, 4:]

def U13_5():
    data = np.arange(1, 499875841)
    chunks = [data[i:i+4] for i in range(len(data)) if (i % 4) == 0]
    a, b = chunks[::2], chunks[1::2]

def U13_6():
    data = np.arange(1, 499875841)
    chunks = [[data[i:i+4], data[i+4:i+8]] for i in range(len(data)) if (i % 8) == 0]
    a, b = zip(*chunks)

def hpaulj():
    # Generalized
    data = np.arange(1, 499875841)
    a,b,c = arr.reshape(-1, M, N).transpose(1, 0, 2).reshape(-1, arr.size//M) 

def mozway():
    # Generalized
    data = np.arange(1, 499875841)
    out = np.split(np.argsort(np.arange(len(data))%(4*2)//4, kind='stable'), 2)

def pauls():
    data = np.arange(1, 499875841)
    i = np.arange(1, len(data)+1) % 8 
    m = (i >= 1) & (i <= 4) 
    a = data[m]
    b = data[~m]

U13_1 = timeit.timeit('U13_1()', 'from __main__ import U13_1', number=1)
print('U13_1 Speed:', U13_1)
U13_2 = timeit.timeit('U13_2()', 'from __main__ import U13_2', number=1)
print('U13_2 Speed:', U13_2)
U13_3 = timeit.timeit('U13_3()', 'from __main__ import U13_3', number=1)
print('U13_3 Speed:', U13_3)
U13_4 = timeit.timeit('U13_4()', 'from __main__ import U13_4', number=1)
print('U13_4 Speed:', U13_4)
U13_5 = timeit.timeit('U13_5()', 'from __main__ import U13_5', number=1)
print('U13_5 Speed:', U13_5)
hpaulj = timeit.timeit('hpaulj()', 'from __main__ import hpaulj', number=1)
print('U13_6 Speed:', U13_6)
hpaulj = timeit.timeit('hpaulj()', 'from __main__ import hpaulj', number=1)
print('hpaulj Speed:', hpaulj)
mozway = timeit.timeit('mozway()', 'from __main__ import mozway', number=1)
print('mozway Speed:', mozway)
pauls = timeit.timeit('pauls()', 'from __main__ import pauls', number=1)
print('pauls Speed:', pauls)

import matplotlib.pyplot as plt
df = pd.DataFrame({'U13_1': U13_1, 'U13_2': U13_2, 'U13_3': U13_3, 'U13_4': U13_4, 'U13_5': U13_5, 'U13_6': U13_6, 'hpaulj': hpaulj, 'mozway': mozway, 'pauls': pauls}, index=[0])

plt.plot(df.columns, df.iloc[0])
plt.show()

U13_1 Speed: 0.45211420000032376
U13_2 Speed: 378.43887300000006
U13_3 Speed: 0.6713313000000198
U13_4 Speed: 0.6900013000004037
U13_5 Speed: 298.6191995999998
U13_6 Speed: 601.4342458000001
hpaulj Speed: 1.2215797999997449
mozway Speed: 21.53476950000004
pauls Speed: 4.3681056999994325

My solution #1 Generalized
My solution #3 Not Generalized
My solution #4 Not Generalized

My solution #1 Generalized
@hpaulj's solution Generalized
@mozway's solution Generalized

added 1099 characters in body

Source Link

edited Jun 14, 2024 at 10:14

U13-Forward

71.9k
15
100
126

TL;DR, for fastest solution, go to Solution #4.

Solution 1: Fastest oneReshape and chunking

Solution 4: Fastest one + Generalized:

We reshape the matrix to already done chunks, and then we use a list comprehension to slice the arrays and accomplish the task.

I use reshaped[:, i, :] to filter out the specific chunk from the 3D matrix.

Code below:

reshaped = data.reshape(-1, n, m)
chunks = [reshaped[:, i, :] for i in range(n)]

Solution 45: Using modulo for conditional slicing in list comprehension

Solution 56: Using list comprehension and zip:

I'm testing the following array 1,000of exactly 499875840 items,000 times as the OP specified to seedetermine the speed of each solution:

data = np.arrayarange([1,2,3,4,5,6,7,8,9,10,11,12,13,14,151,16] 499875841)

import timeit

def U13_1():
    data = np.arrayarange([1,2,3,4,5,6,7,8,9,10,11,12,13,14,151,16] 499875841)
    reshaped = data.reshape(-1, 8)
    a = reshaped[:, :4]
    b = reshaped[:, 4:]

def U13_2():
    data = np.arrayarange([1,2,3,4,5,6,7,8,9,10,11,12,13,14,151,16] 499875841)
    chunks = np.array_split(data, (data.shape[0] + 3) // 4)
    a, b = chunks[::2], chunks[1::2]

def U13_3():
    data = np.arrayarange([1,2,3,4,5,6,7,8,9,10,11,12,13,14,151,16] 499875841)
    chunks = data.reshape((data.shape[0] // 4, 4))
    a, b = chunks[::2], chunks[1::2]

def U13_4():
    # Generalized
    data = np.arrayarange([1,2,3,4,5,6,7,8,9,101,11 499875841)
    reshaped = data.reshape(-1,12 n,13 m)
    chunks = [reshaped[:,14 i,15 :] for i in range(n)]

def U13_5():
    data = np.arange(1,16] 499875841)
    chunks = [data[i:i+4] for i in range(len(data)) if (i % 4) == 0]
    a, b = chunks[::2], chunks[1::2]

def U13_5U13_6():
    data = np.arrayarange([1,2,3,4,5,6,7,8,9,10,11,12,13,14,151,16] 499875841)
    chunks = [[data[i:i+4], data[i+4:i+8]] for i in range(len(data)) if (i % 8) == 0]
    a, b = zip(*chunks)

def hpaulj():
    # Generalized
    data = np.arrayarange([1,2,3,4,5,6,7,8,9,10,11,12,13,14,151,16] 499875841)
    a, b,c = dataarr.reshape(-1,2 M,4 N).transpose(1, 0, 2).reshape(-1, arr.size//M) 

def mozway():
    data =# np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16])Generalized
    Ndata = 4
    M =np.arange(1, 2499875841)
    out = np.split(np.argsort(np.arange(len(data))%(N*M4*2)//N4, kind='stable'), M2)

def pauls():
    data = np.arrayarange([1,2,3,4,5,6,7,8,9,10,11,12,13,14,151,16] 499875841)
    i = np.arange(1, len(data)+1) % 8 
    m = (i >= 1) & (i <= 4) 
    a = data[m]
    b = data[~m]

print('U13_1U13_1 Speed:',= timeit.timeit('U13_1()', 'from __main__ import U13_1', number=1000000)number=1)
print('U13_2'U13_1 Speed:', U13_1)
U13_2 = timeit.timeit('U13_2()', 'from __main__ import U13_2', number=1000000)number=1)
print('U13_3'U13_2 Speed:', U13_2)
U13_3 = timeit.timeit('U13_3()', 'from __main__ import U13_3', number=1000000)number=1)
print('U13_4'U13_3 Speed:', U13_3)
U13_4 = timeit.timeit('U13_4()', 'from __main__ import U13_4', number=1000000)number=1)
print('U13_5'U13_4 Speed:', U13_4)
U13_5 = timeit.timeit('U13_5()', 'from __main__ import U13_5', number=1000000)number=1)
print('hpaulj'U13_5 Speed:', U13_5)
hpaulj = timeit.timeit('hpaulj()', 'from __main__ import hpaulj', number=1000000number=1)
print('U13_6 Speed:', U13_6)
hpaulj = timeit.timeit('hpaulj()', 'from __main__ import hpaulj', number=1)
print('mozway'hpaulj Speed:', hpaulj)
mozway = timeit.timeit('mozway()', 'from __main__ import mozway', number=1000000)number=1)
print('pauls'mozway Speed:', mozway)
pauls = timeit.timeit('pauls()', 'from __main__ import pauls', number=1000000number=1)
print('pauls Speed:', pauls)

import matplotlib.pyplot as plt
df = pd.DataFrame({'U13_1': U13_1, 'U13_2': U13_2, 'U13_3': U13_3, 'U13_4': U13_4, 'U13_5': U13_5, 'U13_6': U13_6, 'hpaulj': hpaulj, 'mozway': mozway, 'pauls': pauls}, index=[0])

plt.plot(df.columns, df.iloc[0])
plt.show()

U13_1 Speed: 10.53733970000757846900013000004037
U13_2 Speed: 10378.71971810000832243887300000006
U13_3 Speed: 20.62628799999947666713313000000198
U13_4 Speed: 30.9350442000140945211420000032376
U13_5 Speed: 4298.272379000001826191995999998
U13_6 Speed: 601.4342458000001
hpaulj Speed: 31.35782740000286142215797999997449
mozway Speed: 1921.90946940000867553476950000004
pauls Speed: 94.7448797000106423681056999994325

Fastest ones:

My solution #1#4 Generalized
My solution #2#3 Not Generalized
@hpualj'sMy solution #1 Not Generalized

Fastest ones (only including generalized):

My solution #4 Generalized

@hpaulj's solution Generalized

@mozway's solution Generalized

Solution 1: Fastest one

Solution 4: Using modulo for conditional slicing in list comprehension

Solution 5: Using list comprehension and zip:

I'm testing the following array 1,000,000 times to see the speed of each solution:

data = np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16])

import timeit

def U13_1():
    data = np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16])
    reshaped = data.reshape(-1, 8)
    a = reshaped[:, :4]
    b = reshaped[:, 4:]

def U13_2():
    data = np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16])
    chunks = np.array_split(data, (data.shape[0] + 3) // 4)
    a, b = chunks[::2], chunks[1::2]

def U13_3():
    data = np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16])
    chunks = data.reshape((data.shape[0] // 4, 4))
    a, b = chunks[::2], chunks[1::2]

def U13_4():
    data = np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16])
    chunks = [data[i:i+4] for i in range(len(data)) if (i % 4) == 0]
    a, b = chunks[::2], chunks[1::2]

def U13_5():
    data = np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16])
    chunks = [[data[i:i+4], data[i+4:i+8]] for i in range(len(data)) if (i % 8) == 0]
    a, b = zip(*chunks)

def hpaulj():
    data = np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16])
    a, b = data.reshape(-1,2,4).transpose(1,0,2)

def mozway():
    data = np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16])
    N = 4
    M = 2
    out = np.split(np.argsort(np.arange(len(data))%(N*M)//N, kind='stable'), M)

def pauls():
    data = np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16])
    i = np.arange(1, len(data)+1) % 8 
    m = (i >= 1) & (i <= 4) 
    a = data[m]
    b = data[~m]

print('U13_1 Speed:', timeit.timeit('U13_1()', 'from __main__ import U13_1', number=1000000))
print('U13_2 Speed:', timeit.timeit('U13_2()', 'from __main__ import U13_2', number=1000000))
print('U13_3 Speed:', timeit.timeit('U13_3()', 'from __main__ import U13_3', number=1000000))
print('U13_4 Speed:', timeit.timeit('U13_4()', 'from __main__ import U13_4', number=1000000))
print('U13_5 Speed:', timeit.timeit('U13_5()', 'from __main__ import U13_5', number=1000000))
print('hpaulj Speed:', timeit.timeit('hpaulj()', 'from __main__ import hpaulj', number=1000000))
print('mozway Speed:', timeit.timeit('mozway()', 'from __main__ import mozway', number=1000000))
print('pauls Speed:', timeit.timeit('pauls()', 'from __main__ import pauls', number=1000000))

U13_1 Speed: 1.5373397000075784
U13_2 Speed: 10.719718100008322
U13_3 Speed: 2.6262879999994766
U13_4 Speed: 3.93504420001409
U13_5 Speed: 4.27237900000182
hpaulj Speed: 3.3578274000028614
mozway Speed: 19.909469400008675
pauls Speed: 9.744879700010642

My solution #1
My solution #2
@hpualj's solution

TL;DR, for fastest solution, go to Solution #4.

Solution 1: Reshape and chunking

Solution 4: Fastest one + Generalized:

We reshape the matrix to already done chunks, and then we use a list comprehension to slice the arrays and accomplish the task.

I use reshaped[:, i, :] to filter out the specific chunk from the 3D matrix.

Code below:

reshaped = data.reshape(-1, n, m)
chunks = [reshaped[:, i, :] for i in range(n)]

Solution 5: Using modulo for conditional slicing in list comprehension

Solution 6: Using list comprehension and zip:

I'm testing the following of exactly 499875840 items, as the OP specified to determine the speed of each solution:

data = np.arange(1, 499875841)

import timeit

def U13_1():
    data = np.arange(1, 499875841)
    reshaped = data.reshape(-1, 8)
    a = reshaped[:, :4]
    b = reshaped[:, 4:]

def U13_2():
    data = np.arange(1, 499875841)
    chunks = np.array_split(data, (data.shape[0] + 3) // 4)
    a, b = chunks[::2], chunks[1::2]

def U13_3():
    data = np.arange(1, 499875841)
    chunks = data.reshape((data.shape[0] // 4, 4))
    a, b = chunks[::2], chunks[1::2]

def U13_4():
    # Generalized
    data = np.arange(1, 499875841)
    reshaped = data.reshape(-1, n, m)
    chunks = [reshaped[:, i, :] for i in range(n)]

def U13_5():
    data = np.arange(1, 499875841)
    chunks = [data[i:i+4] for i in range(len(data)) if (i % 4) == 0]
    a, b = chunks[::2], chunks[1::2]

def U13_6():
    data = np.arange(1, 499875841)
    chunks = [[data[i:i+4], data[i+4:i+8]] for i in range(len(data)) if (i % 8) == 0]
    a, b = zip(*chunks)

def hpaulj():
    # Generalized
    data = np.arange(1, 499875841)
    a,b,c = arr.reshape(-1, M, N).transpose(1, 0, 2).reshape(-1, arr.size//M) 

def mozway():
    # Generalized
    data = np.arange(1, 499875841)
    out = np.split(np.argsort(np.arange(len(data))%(4*2)//4, kind='stable'), 2)

def pauls():
    data = np.arange(1, 499875841)
    i = np.arange(1, len(data)+1) % 8 
    m = (i >= 1) & (i <= 4) 
    a = data[m]
    b = data[~m]

U13_1 = timeit.timeit('U13_1()', 'from __main__ import U13_1', number=1)
print('U13_1 Speed:', U13_1)
U13_2 = timeit.timeit('U13_2()', 'from __main__ import U13_2', number=1)
print('U13_2 Speed:', U13_2)
U13_3 = timeit.timeit('U13_3()', 'from __main__ import U13_3', number=1)
print('U13_3 Speed:', U13_3)
U13_4 = timeit.timeit('U13_4()', 'from __main__ import U13_4', number=1)
print('U13_4 Speed:', U13_4)
U13_5 = timeit.timeit('U13_5()', 'from __main__ import U13_5', number=1)
print('U13_5 Speed:', U13_5)
hpaulj = timeit.timeit('hpaulj()', 'from __main__ import hpaulj', number=1)
print('U13_6 Speed:', U13_6)
hpaulj = timeit.timeit('hpaulj()', 'from __main__ import hpaulj', number=1)
print('hpaulj Speed:', hpaulj)
mozway = timeit.timeit('mozway()', 'from __main__ import mozway', number=1)
print('mozway Speed:', mozway)
pauls = timeit.timeit('pauls()', 'from __main__ import pauls', number=1)
print('pauls Speed:', pauls)

import matplotlib.pyplot as plt
df = pd.DataFrame({'U13_1': U13_1, 'U13_2': U13_2, 'U13_3': U13_3, 'U13_4': U13_4, 'U13_5': U13_5, 'U13_6': U13_6, 'hpaulj': hpaulj, 'mozway': mozway, 'pauls': pauls}, index=[0])

plt.plot(df.columns, df.iloc[0])
plt.show()

U13_1 Speed: 0.6900013000004037
U13_2 Speed: 378.43887300000006
U13_3 Speed: 0.6713313000000198
U13_4 Speed: 0.45211420000032376
U13_5 Speed: 298.6191995999998
U13_6 Speed: 601.4342458000001
hpaulj Speed: 1.2215797999997449
mozway Speed: 21.53476950000004
pauls Speed: 4.3681056999994325

Fastest ones:

My solution #4 Generalized
My solution #3 Not Generalized
My solution #1 Not Generalized

Fastest ones (only including generalized):

My solution #4 Generalized

@hpaulj's solution Generalized

@mozway's solution Generalized

added 2814 characters in body

Source Link

edited Jun 13, 2024 at 22:19

U13-Forward

71.9k
15
100
126

Loading

added 1339 characters in body

Source Link

edited Jun 13, 2024 at 21:55

U13-Forward

71.9k
15
100
126

Loading

added 1339 characters in body

Source Link

edited Jun 13, 2024 at 21:49

U13-Forward

71.9k
15
100
126

Loading

added 192 characters in body

Source Link

edited Jun 10, 2024 at 17:46

U13-Forward

71.9k
15
100
126

Loading

added 287 characters in body

Source Link

edited Jun 10, 2024 at 17:37

U13-Forward

71.9k
15
100
126

Loading

added 177 characters in body

Source Link

edited Jun 10, 2024 at 17:27

U13-Forward

71.9k
15
100
126

Loading

Source Link

answered Jun 10, 2024 at 17:15

U13-Forward

71.9k
15
100
126

Loading

Collectives™ on Stack Overflow

Return to Answer

Post Timeline

Solution 1: Fastest one + Generalized:

Solution 4: Fastest one + Generalized:

Solution 4: Fastest one + Generalized:

Solution 1: Fastest one + Generalized:

Solution 4: Fastest one + Generalized:

Solution 4: Fastest one + Generalized: