Skip to content

Commit 299699f

Browse files
committed
Add dev workflow README, improve other readmes
1 parent b587995 commit 299699f

8 files changed

Lines changed: 151 additions & 61 deletions

File tree

‎05-example-web-application/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33
![](./readme-assets/app-screenshot.png)
44

55
## Minimal 3 tier web application
6-
- React frontend
7-
- Node JS and Golang APIs
8-
- Postgres Database
6+
- **React frontend:** Uses react query to load data from the two apis and display the result
7+
- **Node JS and Golang APIs:** Both have `/` and `/ping` endpoints. `/` queries the Database for the current time, and `/ping` returns `pong`
8+
- **Postgres Database:** An empty PostgreSQL database with no tables or data. Used to show how to set up connectivity. The API applications execute `SELECT NOW() as now;` to determine the current time to return.
99

1010
![](./readme-assets/tech-stack.png)
1111

‎10-interacting-with-docker-objects/README.md‎

Lines changed: 81 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -8,45 +8,93 @@ You should:
88

99
## Images
1010

11-
1) ls
12-
2) build (https://docs.docker.com/engine/reference/commandline/build/)
13-
3) tag
14-
4) pull
15-
5) push
16-
6) rm
17-
7) prune
18-
8) save
19-
9) docker scan <image> (snyk security scan, also show trivy)
11+
`docker image COMMAND`:
12+
```
13+
build Build an image from a Dockerfile (`docker build` is the same as `docker image build`)
14+
history Show the history of an image
15+
import Import the contents from a tarball to create a filesystem image
16+
inspect Display detailed information on one or more images
17+
load Load an image from a tar archive or STDIN
18+
ls List images
19+
prune Remove unused images
20+
pull Pull an image or a repository from a registry
21+
push Push an image or a repository to a registry
22+
rm Remove one or more images
23+
save Save one or more images to a tar archive (streamed to STDOUT by default)
24+
tag Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
25+
```
26+
27+
### Scanning Images
28+
29+
Not a `docker image` subcommand, but still something you do with images:
30+
31+
```
32+
docker scan IMAGE
33+
```
34+
35+
***Note:*** You can also use a 3rd party scanner such as Trivy (https://github.com/aquasecurity/trivy)
36+
37+
### Signing Images
38+
39+
Another protection against software supply chain attacks is the ability to uniquely sign specific image tags to ensure an image was created by the entity who signed it.
40+
41+
```
42+
docker trust sign IMAGE:TAG
43+
docker trust inspect --pretty IMAGE:TAG
44+
```
2045

2146
## Containers
2247

23-
1) ls
24-
2) run
25-
3) start
26-
4) attach
27-
5) exec
28-
6) logs
29-
7) top
30-
8) cp
31-
9) stop
32-
10) kill
33-
11) prune
34-
12) export
48+
`docker container COMMAND`:
49+
50+
```
51+
attach Attach local standard input, output, and error streams to a running container
52+
commit Create a new image from a container's changes
53+
cp Copy files/folders between a container and the local filesystem
54+
create Create a new container
55+
diff Inspect changes to files or directories on a container's filesystem
56+
exec Run a command in a running container
57+
export Export a container's filesystem as a tar archive
58+
inspect Display detailed information on one or more containers
59+
kill Kill one or more running containers
60+
logs Fetch the logs of a container
61+
ls List containers
62+
pause Pause all processes within one or more containers
63+
port List port mappings or a specific mapping for the container
64+
prune Remove all stopped containers
65+
rename Rename a container
66+
restart Restart one or more containers
67+
rm Remove one or more containers
68+
run Run a command in a new container
69+
start Start one or more stopped containers
70+
stats Display a live stream of container(s) resource usage statistics
71+
stop Stop one or more running containers
72+
top Display the running processes of a container
73+
unpause Unpause all processes within one or more containers
74+
update Update configuration of one or more containers
75+
wait Block until one or more containers stop, then print their exit codes
76+
```
3577

3678
## Volumes
3779

38-
1) ls
39-
2) create
40-
3) inspect
41-
4) rm
42-
5) prune
80+
`docker volume COMMAND`:
81+
```
82+
create Create a volume
83+
inspect Display detailed information on one or more volumes
84+
ls List volumes
85+
prune Remove all unused local volumes
86+
rm Remove one or more volumes
87+
```
4388

4489
## Networks
4590

46-
1) ls
47-
2) create
48-
3) inspect
49-
4) connect
50-
5) disconnect
51-
6) rm
52-
7) prune
91+
`docker network COMMAND`:
92+
```
93+
connect Connect a container to a network
94+
create Create a network
95+
disconnect Disconnect a container from a network
96+
inspect Display detailed information on one or more networks
97+
ls List networks
98+
prune Remove all unused networks
99+
rm Remove one or more networks
100+
```

‎11-development-workflow/Makefile‎

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
DEV_COMPOSE_FILE=docker-compose-dev.yml
22
DEBUG_COMPOSE_FILE=docker-compose-debug.yml
3+
TEST_COMPOSE_FILE=docker-compose-test.yml
34

45
### DOCKER COMPOSE COMMANDS
56

@@ -26,14 +27,9 @@ compose-down:
2627
###
2728

2829
DOCKERCONTEXT_DIR:=../05-example-web-application/
29-
DOCKERFILE_DIR:=../10-development-workflow/
30-
31-
.PHONY: docker-build-all
32-
docker-build-all:
33-
docker build -t api-node -f ${DOCKERFILE_DIR}/api-node/Dockerfile.dev ${DOCKERCONTEXT_DIR}/api-node/
34-
docker build -t api-golang -f ${DOCKERFILE_DIR}/api-golang/Dockerfile.dev ${DOCKERCONTEXT_DIR}/api-golang/
30+
DOCKERFILE_DIR:=../11-development-workflow/
3531

3632
.PHONY: run-tests
3733
run-tests:
38-
docker run -t api-golang go test -v ./...
39-
docker run -it api-node npm run test
34+
docker compose -f $(DEV_COMPOSE_FILE) -f $(TEST_COMPOSE_FILE) run --build api-golang
35+
docker compose -f $(DEV_COMPOSE_FILE) -f $(TEST_COMPOSE_FILE) run --build api-node

‎11-development-workflow/README.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Development Workflow
2+
3+
## Development Environment
4+
5+
Because we are running our application within containers, we need a way to quickly iterate and make changes to them. Some of our tactics in `06-building-container-images` help here (e.g. protecting the layer cache) so that images build quickly, but we can do better.
6+
7+
We want our development environment to have the following attributes:
8+
9+
1) **Easy/simple to set up:** Using docker compose, we can define the entire environment with a single yaml file. To get started, team members can issue a single command `make compose-up-build` or `make compose-up-build-debug` depending if they want to run the debugger or not.
10+
11+
2) **Ability to iterate without rebuilding the container image:** In order to avoid having to rebuild the container image with every single change, we can use a bind mount to mount the code from our host into the container filesystem. For example:
12+
13+
```yml
14+
- type: bind
15+
source: ../05-example-web-application/api-node/
16+
target: /usr/src/app/
17+
```
18+
19+
3) **Automatic reloading of the application:**
20+
- <ins>*React Client:*</ins> We are using Vite for the react client which handles this handles this automatically
21+
- <ins>*Node API:*</ins> We added nodemon as a development dependency and specify the Docker CMD to use it
22+
- <ins>*Golang API:*</ins> We added a utility called `air` (https://github.com/cosmtrek/air) within `Dockerfile.dev` which watches for changes and rebuild the app automatically.
23+
24+
4) **Use a debugger:**
25+
- <ins>*React Client:*</ins> For a react app, you can use the browser developer tools + extensions to debug. I did include `react-query-devtools` to help debug react query specific things. It is also viewed from within the browser.
26+
- <ins>*Node API:*</ins> To enable debugging for a NodeJS application we can run the app with the `--inspect` flag. The debug session can then be accessed via a websocket on port `9229`. The additional considerations in this case are to specify that the debugger listen for requests from 0.0.0.0 (any) and to publish port `9229` from the container to localhost.
27+
- <ins>*Golang API:*</ins> To enable remote debugging for a golang application I installed a tool called delve (https://github.com/go-delve/delve) within `./api-golang/Dockerfile.dev`. We then override the command used to run the container to use this tool (see: `docker-compose-debug.yml`)
28+
29+
---
30+
31+
These modifications to the configuration (overridden commands + port publishing) are specified in `docker-compose-debug.yml`. By passing both `docker-compose-dev.yml` AND `docker-compose-debug.yml` to the `docker compose up` command (See: `make compose-up-debug-build`) Docker combines the two files, taking the config from the latter and overlaying it onto the former.
32+
33+
Both `./api-golang/README.md` and `./api-node/README.md` show a launch.json configuration you can use to connnect to these remote debuggers using VSCode. The key setting is `substitutePath` such that you can set breakpoints on your local system that get recognized within the container.
34+
35+
5) **Executing tests:** We also need the ability to execute our test suites within containers. Again, we can create a custom `docker-compose-test.yml` overlay which modifies the container commands to execute our tests. To build the api images and execute their tests, you can execute `make run-tests` which will use the `test` compose file along with the `dev` compose file to do so.
36+
37+
## Continuous Integration
38+
39+
See `.github/workflows/image-ci.yml` for a basic GitHub Action workflow that builds, scans, tags, and pushes a container image.
40+
41+
It leverages a few publicly available actions from the marketplace:
42+
1) https://github.com/marketplace/actions/docker-metadata-action (generates tags for the container images)
43+
2) https://github.com/marketplace/actions/docker-login (logs into DockerHub)
44+
3) https://github.com/marketplace/actions/build-and-push-docker-images (builds and pushes the images)
45+
4) https://github.com/marketplace/actions/aqua-security-trivy (scans the images for vulnerabilities)
46+
47+
If you want to build out more advanced CI workflows I recommend looking at Bret Fisher's `Automation with Docker for CI/CD Workflows` repo (https://github.com/BretFisher/docker-cicd-automation). It has many great examples of the types of things you might want to do with Docker in a CI/CD pipeline!

‎11-development-workflow/api-golang/Dockerfile.dev‎

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,4 @@
1-
# Pin specific version for stability
2-
# using bullseye instead of alpine because of:
3-
## runtime/cgo
4-
## cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH
1+
# Using bullseye instead of alpine because debugger didnt work in alpine
52
FROM golang:1.19-bullseye
63

74
WORKDIR /app
@@ -12,7 +9,6 @@ RUN go install github.com/cosmtrek/air@latest
129
# Install delve for debugging
1310
RUN go install github.com/go-delve/delve/cmd/dlv@latest
1411

15-
# Copy only files required to install dependencies (better layer caching)
1612
COPY go.mod go.sum ./
1713

1814
RUN go mod download
Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,15 @@
1-
# Pin specific version for stability
2-
# Use alpine for reduced image size
31
FROM node:19.4-alpine as dev
42

5-
# Specify working directory other than /
63
WORKDIR /usr/src/app
74

8-
# Copy only files required to install
9-
# dependencies (better layer caching)
105
COPY package*.json ./
116

12-
# Install only production dependencies
13-
# Use cache mount to speed up install of existing dependencies
147
RUN --mount=type=cache,target=/usr/src/app/.npm \
158
npm set cache /usr/src/app/.npm && \
169
npm install
1710

18-
# Copy remaining source code AFTER installing dependencies.
19-
# Again, copy only the necessary files
2011
COPY . .
2112

22-
# Indicate expected port
2313
EXPOSE 3000
2414

2515
CMD [ "npm", "run", "dev" ]

‎11-development-workflow/docker-compose-dev.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ services:
1818
api-node:
1919
build:
2020
context: ../05-example-web-application/api-node/
21-
dockerfile: ../../10-development-workflow/api-node/Dockerfile.dev
21+
dockerfile: ../../11-development-workflow/api-node/Dockerfile.dev
2222
target: dev
2323
volumes:
2424
- type: bind
@@ -37,7 +37,7 @@ services:
3737
api-golang:
3838
build:
3939
context: ../05-example-web-application/api-golang/
40-
dockerfile: ../../10-development-workflow/api-golang/Dockerfile.dev
40+
dockerfile: ../../11-development-workflow/api-golang/Dockerfile.dev
4141
volumes:
4242
- type: bind
4343
source: ../05-example-web-application/api-golang/
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Overlay configuration to enable debuggers
2+
services:
3+
api-node:
4+
command:
5+
- "npm"
6+
- "run"
7+
- "test"
8+
api-golang:
9+
command:
10+
- "go"
11+
- "test"
12+
- "-v"
13+
- "./..."

0 commit comments

Comments
 (0)