EDIT: added much more detailed example, see below.

Scenario

we have 3 groups/copies of of following:

(figure illustrates 1 group)

Everything is flattened, so emb of 3 src node, with emb_size=32, is torch.Size([3, 32]). And, emb of 6 tgt node torch.Size([6, 32])

Goal

We want to reshape the data so that each src corresponds to 2 tgt node, so we do:

Now, for i-th src node, we have:

• source_embs[i,:]
• with the corresponding target_embs[i,:,:]
• This is the whole point: data is now neatly organized, without reshaping we cant do this simple indexing.

Details

Looking at shape of target_embs:

• before reshaping, shape is [6,32]
• we start from rightmost dim, dim1=32, it isn't changed in the reshape, so ignore
• we view shape as [6,*], and now the rightmost dim is dim0=6, almost like ignore dim1, and view it as [6]
• When we reshape [6] into [3,2], we always look at the rightmost dim first, so we take 2 elements, then change row, then 2 element then change row and so on
• As prior knowledge, we know [6,*] corresponds to [src1_tgt1, src1_tgt2, src2_tgt1, src2_tgt2, src3_tgt1, src3_tgt2] (this input has to be in this format, or else we need to rearrange the input into this format)
• hence we know output is formatted correctly: [3,2] will correspond to what we want: [[src1_tgt1,src1_tgt2],[src2_tgt1, src2_tgt2],[src3_tgt1, src3_tgt2]]
• So reshaping [6,32] into [3,2,32] is now complete
• what if we want to reshape [6,32] into [4,3,16]? torch can do this, because the index match up, but the result is useless to our purposes
• what if we want to have [32,2,3] in the end instead of [3,2,32]? Do we just do reshape(input6x32,(32,2,3))? No. Because the data will be scrambled and will be meaningless. What we can do is to get to [3,2,32] first, and then use transpose() into [32,2,3].

summary (for basic usage)

• reshape 2 consecutive dimensions at a time, and only 2. This way it's much more understandable.
• If want to reshape non-consecutive dimensions, then transpose before reshaping
• There probably are more advanced usages, but this is the only way I manage to understand what reshape() is doing.

I didn't manage to understand what np.reshape() does until I read this article.

Mechanically it is clear what reshape() does. But how do we interpret the data before and after reshape?

The missing piece for me was:

When we train a machine learning model, the nesting levels of arrays have precisely defined meaning.

This means that the reshape operation has to be keenly aware both points below before the operation has any meaning:

• the data it operates on (how the reshape intput looks like)
• how the algorithm/model expects the reshaped data to be (how the reshape output looks like)

For example:

The external array contains observations/rows. The inner array contains columns/features. This causes two special cases when we have either an array of multiple observations of only one feature or a single observation of multiple features.

For more advanced example: See this stackoverflow question

I didn't manage to understand what np.reshape() does until I read this article.

Mechanically it is clear what reshape() does. But how do we interpret the data before and after reshape?

The missing piece for me was:

When we train a machine learning model, the nesting levels of arrays have precisely defined meaning.

This means that the reshape operation has to be keenly aware both points below before the operation has any meaning:

• the data it operates on (how the reshape input looks like)
• how the algorithm/model expects the reshaped data to be (how the reshape output looks like)

For example:

The external array contains observations/rows. The inner array contains columns/features. This causes two special cases when we have either an array of multiple observations of only one feature or a single observation of multiple features.

For more advanced example: See this stackoverflow question