Tensorflow, Keras, MXNet, PyTorch
If the neural network is given as a Tensorflow graph, then you can visualize this graph with TensorBoard.
Here is how the MNIST CNN looks like:
You can add names / scopes (like "dropout", "softmax", "fc1", "conv1", "conv2") yourself.
Interpretation
The following is only about the left graph. I ignore the 4 small graphs on the right half.
Each box is a layer with parameters that can be learned. For inference, information flows from bottom to the top. Ellipses are layers which do not contain learned parameters.
The color of the boxes does not have a meaning.
I'm not sure of the value of the dashed small boxes ("gradients", "Adam", "save").