Compare commits
3 Commits
main
...
docker_mem
| Author | SHA1 | Date | |
|---|---|---|---|
|
d8058c901c
|
||
|
|
2cf03e2c3b
|
||
|
|
f98dccf92e
|
@ -1,24 +0,0 @@
|
||||
#+OPTIONS: html-postamble:nil
|
||||
#+date: <2023-12-23 Sat>
|
||||
#+author: Tom Alexander
|
||||
#+email:
|
||||
#+language: en
|
||||
#+select_tags: export
|
||||
#+exclude_tags: noexport
|
||||
|
||||
My dev blog will appear here as soon as I finish writing articles worthy of publishing. In the mean time, please check out my repos at [[https://code.fizz.buzz/explore/repos][code.fizz.buzz]].
|
||||
|
||||
Links:
|
||||
- My personal repos: [[https://code.fizz.buzz/explore/repos][code.fizz.buzz]]
|
||||
- LinkedIn: https://www.linkedin.com/in/tom-alexander-b6a18216/
|
||||
- GitHub: https://github.com/tomalexander
|
||||
- Resume: https://fizz.buzz/tom_alexander_resume.pdf
|
||||
- PGP Key: https://fizz.buzz/pgp.asc
|
||||
|
||||
* Why is your website the way it is?
|
||||
|
||||
I used to have a developer blog hosted at this domain. I quickly developed an appreciation for the power of org-mode for writing the content of the blog but I grew tired of inconsistent build results. The static site generators at the time would function by calling out to emacs itself to parse the org-mode and export HTML which meant that updates to emacs, my elisp packages, or the static site generator could cause compatibility issues. This often lead to things like escaping issues in old blog posts going unnoticed.
|
||||
|
||||
To solve the issue, and to seize the opportunity to gain more experience in Rust, I decided to write my own static site generator that would not depend on outside tools. So far I have written [[https://code.fizz.buzz/talexander/duster][the template engine]] and I am in the process of writing [[https://code.fizz.buzz/talexander/organic][an org-mode parser]]. When that is done, it should just be a matter of tying those two together with some minor glue to make a static site generator to create the new version of this site. Until that is done, I am using this hastily thrown-together manually-written html file as a placeholder.
|
||||
|
||||
That isn't to say that there are no exciting things hosted on this server, just not at the root domain. For example, this server is running kubernetes that I set up manually following [[https://github.com/kelseyhightower/kubernetes-the-hard-way][kubernetes-the-hard-way]] in a bunch of [[https://man.freebsd.org/cgi/man.cgi?bhyve][bhyve VMs]] that I networked together using [[https://man.freebsd.org/cgi/man.cgi?netgraph(4)][netgraph]]. On it I host my own [[https://www.powerdns.com/][PowerDNS]] server as the authoratative DNS server for fizz.buzz. It is integrated with [[https://cert-manager.io/][cert-manager]] and [[https://github.com/kubernetes-sigs/external-dns][ExternalDNS]] so Ingresses/LoadBalancers on my cluster automatically get valid TLS certificates and update the DNS records. I have a fully open-source self-hosted gitops workflow where a commit to a git repo I'm hosting in [[https://code.fizz.buzz/][gitea]], triggers a [[https://tekton.dev/][tekton pipeline]] through [[https://github.com/jenkins-x/lighthouse][lighthouse]] to build a docker image with [[https://github.com/GoogleContainerTools/kaniko][kaniko]], which gets pushed to my self-hosted [[https://goharbor.io/][harbor]] instance, which then gets deployed to my cluster via [[https://fluxcd.io/][flux]]. The end result is I make a commit to a repo and the result is deployed to my website in minutes.
|
||||
497
posts/2024/10/18/graph-docker-memory/index.org
Normal file
497
posts/2024/10/18/graph-docker-memory/index.org
Normal file
@ -0,0 +1,497 @@
|
||||
#+OPTIONS: html-postamble:nil
|
||||
#+title: Graph Docker Container Memory Usage with Gnuplot
|
||||
#+date: <2024-10-18 Fri>
|
||||
#+author: Tom Alexander
|
||||
#+email:
|
||||
#+language: en
|
||||
#+select_tags: export
|
||||
#+exclude_tags: noexport
|
||||
|
||||
Sometimes you need to know how much memory your docker containers are using. Wouldn't it be nice to be able to view their memory usage on a graph? Lets build that.
|
||||
|
||||
* Get the data from docker
|
||||
First, we're going to need a docker container to be running so we can read how much memory it is using. You can use any container you want, but personally I used:
|
||||
#+begin_src bash
|
||||
docker run --rm -i -t alpine:3.20
|
||||
#+end_src
|
||||
|
||||
Now in another terminal, we want to run the src_bash[:exports code]{docker stats} command to see the stats of all running containers. This gets the output:
|
||||
#+begin_src text
|
||||
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
|
||||
1165735d5e55 sharp_chaplygin 0.00% 984KiB / 90.17GiB 0.00% 4.63kB / 0B 1.01MB / 0B 1
|
||||
#+end_src
|
||||
|
||||
That's great, but we want to read this in a script so we can graph the memory usage over time. Our current src_bash[:exports code]{docker stats} command has a couple of problems:
|
||||
1. We don't want it to continuously run. We want a single snapshot of the memory and then the program should exit.
|
||||
2. It would be better we didn't have to parse this bespoke format.
|
||||
3. Personally I'd rather the command didn't truncate the container ID. That is a convenience for human users, but our script can handle the full ID and we can choose to truncate it in our scripts later if we want.
|
||||
|
||||
We can solve that with a couple additional flags:
|
||||
#+begin_src bash
|
||||
$ docker stats --no-stream --no-trunc --format json | jq
|
||||
{
|
||||
"BlockIO": "1.01MB / 0B",
|
||||
"CPUPerc": "0.00%",
|
||||
"Container": "1165735d5e55",
|
||||
"ID": "1165735d5e558652cb70345367dca725e3c48e5a9b6e1aba772f4496274b1fea",
|
||||
"MemPerc": "0.00%",
|
||||
"MemUsage": "984KiB / 90.17GiB",
|
||||
"Name": "sharp_chaplygin",
|
||||
"NetIO": "4.84kB / 0B",
|
||||
"PIDs": "1"
|
||||
}
|
||||
#+end_src
|
||||
|
||||
* Read the stats in python
|
||||
We are going to be generating a graph of this data using [[http://www.gnuplot.info/][gnuplot]]. For the user's convenience, we are going to generate a single gnuplot file that contains the gnuplot commands *and* the data. In our gnuplot file, all the stats for one container will exist in the file, followed by a separataor, and then all the stats for the next docker container. This means, we are going to buffer all of the metrics in memory and write them out at the end rather than streaming them out to the file as we go. So we need to define a class that will record our samples:
|
||||
|
||||
#+begin_src python
|
||||
#!/usr/bin/env python
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import NewType
|
||||
from datetime import datetime
|
||||
|
||||
ContainerId = NewType("ContainerId", str)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Sample:
|
||||
instant: datetime
|
||||
stats: dict[ContainerId, Stats]
|
||||
|
||||
|
||||
@dataclass
|
||||
class Stats:
|
||||
memory_usage_bytes: int
|
||||
#+end_src
|
||||
|
||||
Each time we run src_bash[:exports code]{docker stats} we are going to generate a single src_python[:exports code]{Sample} which will save the memory usage for every running container at that instant. Lets add code that does that:
|
||||
|
||||
#+begin_src python
|
||||
import logging
|
||||
from typing import Tuple
|
||||
import subprocess
|
||||
import json
|
||||
|
||||
ContainerName = NewType("ContainerName", str)
|
||||
|
||||
|
||||
def main():
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
samples: list[Sample] = []
|
||||
labels: dict[ContainerId, ContainerName] = {}
|
||||
print(take_sample())
|
||||
|
||||
|
||||
def take_sample() -> Tuple[Sample, dict[ContainerId, ContainerName]]:
|
||||
labels: dict[ContainerId, ContainerName] = {}
|
||||
stats: dict[ContainerId, Stats] = {}
|
||||
docker_inspect = subprocess.run(
|
||||
["docker", "stats", "--no-stream", "--no-trunc", "--format", "json"],
|
||||
stdout=subprocess.PIPE,
|
||||
)
|
||||
for container_stat in (
|
||||
json.loads(l) for l in docker_inspect.stdout.decode("utf8").splitlines()
|
||||
):
|
||||
if not container_stat["ID"]:
|
||||
# When containers are starting up, they sometimes have no ID and "--" as the name.
|
||||
continue
|
||||
labels[ContainerId(container_stat["ID"])] = ContainerName(
|
||||
container_stat["Name"]
|
||||
)
|
||||
memory_usage = parse_mem_usage(container_stat["MemUsage"])
|
||||
stats[ContainerId(container_stat["ID"])] = Stats(
|
||||
memory_usage_bytes=memory_usage
|
||||
)
|
||||
|
||||
return Sample(instant=datetime.now(), stats=stats), labels
|
||||
|
||||
|
||||
def parse_mem_usage(mem_usage: str) -> int:
|
||||
# TODO: We will implement this.
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
#+end_src
|
||||
|
||||
src_bash[:exports code]{docker stats --format json} outputs data in a jsonl format which means there is one json blob per line. This code invokes src_bash[:exports code]{docker stats}, then iterates over each line from ~stdout~, parsing it as json. It then uses that data to record two values: first it remembers a mapping of container ID to container name, and then it records the memory usage. Finally it returns the src_python[:exports code]{Sample} object.
|
||||
|
||||
Unfortunately, despite exporting in a machine-readable format, src_bash[:exports code]{docker stats} still uses human-readable values for the memory like ~"984KiB / 90.17GiB"~. This means we still need to parse that data. We are going to use regular expressions to grab the constant and the unit, and then we are going to use the unit to convert to the number of bytes:
|
||||
|
||||
#+begin_src python
|
||||
import re
|
||||
|
||||
def parse_mem_usage(mem_usage: str) -> int:
|
||||
parsed_mem_usage = re.match(
|
||||
r"(?P<number>[0-9]+\.?[0-9]*)(?P<unit>[^\s]+)", mem_usage
|
||||
)
|
||||
if parsed_mem_usage is None:
|
||||
raise Exception(f"Invalid Mem Usage: {mem_usage}")
|
||||
number = float(parsed_mem_usage.group("number"))
|
||||
unit = parsed_mem_usage.group("unit")
|
||||
for multiplier, identifier in enumerate(["B", "KiB", "MiB", "GiB", "TiB"]):
|
||||
if unit == identifier:
|
||||
return int(number * (1024**multiplier))
|
||||
raise Exception(f"Unrecognized unit: {unit}")
|
||||
#+end_src
|
||||
|
||||
And finally, we can run our script to get a single sample:
|
||||
|
||||
#+begin_src bash
|
||||
$ python graph_docker_memory.py
|
||||
(Sample(instant=datetime.datetime(2024, 10, 22, 17, 31, 25, 529549), stats={'1165735d5e558652cb70345367dca725e3c48e5a9b6e1aba772f4496274b1fea': Stats(memory_usage_bytes=1007616)}), {'1165735d5e558652cb70345367dca725e3c48e5a9b6e1aba772f4496274b1fea': 'sharp_chaplygin'})
|
||||
#+end_src
|
||||
|
||||
* Adding a loop
|
||||
Now that we can read docker memory usage in python, its time to add our business logic. The flow for our program will be:
|
||||
|
||||
1. Wait for any docker containers to exist.
|
||||
2. Record memory every 2 seconds until no docker containers exist.
|
||||
3. Print out the gnuplot file to ~stdout~.
|
||||
|
||||
We will use the empty response from our src_python[:exports code]{take_sample()} function to indicate no containers are running. Lets update our main function. First, wait for any containers to exist:
|
||||
|
||||
#+begin_src python
|
||||
from time import sleep
|
||||
from typing import Final
|
||||
|
||||
SAMPLE_INTERVAL_SECONDS: Final[int] = 2
|
||||
|
||||
def main():
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
samples: list[Sample] = []
|
||||
labels: dict[ContainerId, ContainerName] = {}
|
||||
first_pass = True
|
||||
# First wait for any docker container to exist.
|
||||
while True:
|
||||
sample, labels_in_sample = take_sample()
|
||||
if labels_in_sample:
|
||||
break
|
||||
if first_pass:
|
||||
first_pass = False
|
||||
logging.info("Waiting for a docker container to exist to start recording.")
|
||||
sleep(1)
|
||||
#+end_src
|
||||
|
||||
Then we want to take samples every 2 seconds, updating our map of docker IDs to docker names, and recording their memory usage:
|
||||
|
||||
#+begin_src python
|
||||
# And then record memory until no containers exist.
|
||||
while True:
|
||||
sample, labels_in_sample = take_sample()
|
||||
if not labels_in_sample:
|
||||
break
|
||||
samples.append(sample)
|
||||
labels = {**labels, **labels_in_sample}
|
||||
sleep(SAMPLE_INTERVAL_SECONDS)
|
||||
#+end_src
|
||||
|
||||
Finally, when no containers exist anymore, we want to print out our gnuplot graph:
|
||||
|
||||
#+begin_src python
|
||||
if labels:
|
||||
# Draws a red horizontal line at 32 GiB since that is the memory limit for cloud run.
|
||||
write_plot(
|
||||
samples,
|
||||
labels,
|
||||
)
|
||||
|
||||
from typing import Collection
|
||||
|
||||
def write_plot(
|
||||
samples: Collection[Sample],
|
||||
labels: dict[ContainerId, ContainerName],
|
||||
):
|
||||
# TODO: We will implement this
|
||||
return
|
||||
#+end_src
|
||||
|
||||
* Generate the gnuplot file
|
||||
|
||||
Finally, we need to write out our gnuplot file. First, we print out a header that tells gnuplot information about our graph:
|
||||
|
||||
#+begin_src python
|
||||
def write_plot(
|
||||
samples: Collection[Sample],
|
||||
labels: dict[ContainerId, ContainerName],
|
||||
):
|
||||
print(
|
||||
"""set terminal svg background '#FFFFFF'
|
||||
set title 'Docker Memory Usage'
|
||||
set xdata time
|
||||
set timefmt '%s'
|
||||
set format x '%tH:%tM:%tS'
|
||||
# Please note this is in SI units (base 10), not IEC (base 2). So, for example, this would show a Gigabyte, not a Gibibyte.
|
||||
set format y '%.0s%cB'
|
||||
set datafile separator "|"
|
||||
"""
|
||||
)
|
||||
#+end_src
|
||||
|
||||
This is telling gnuplot to:
|
||||
1. Output an SVG with a white background.
|
||||
2. Sets the title of the graph.
|
||||
3. Sets the x-axis to be time.
|
||||
4. Sets the input format for the x-axis data to unix timestamps (number of seconds since Jan 1st 1970 in UTC).
|
||||
5. Sets the display format for the x-axis to be Hours:Minutes:Seconds using relative time (data starts at "0").
|
||||
6. Sets the y-axis format to use SI-units (base 10, 1GB = 1,000,000,000 bytes) for bytes.
|
||||
7. Tells gnuplot that we will use the pipe character to separate values in our data.
|
||||
|
||||
Since we're using relative time, we're going to need to know when each container started so we can normalize the timestamps to starting at 0:
|
||||
#+begin_src python
|
||||
starting_time_per_container = {
|
||||
container_id: min(
|
||||
(sample.instant for sample in samples if container_id in sample.stats)
|
||||
)
|
||||
for container_id in labels.keys()
|
||||
}
|
||||
#+end_src
|
||||
|
||||
Now, we need to tell gnuplot about each line (each docker container). The output will look like:
|
||||
#+begin_src text
|
||||
"-" using 1:2 title 'my-first-container' with lines, \
|
||||
"-" using 1:2 title 'my-second-container' with lines
|
||||
#+end_src
|
||||
|
||||
Which is telling gnuplot to read the data from ~stdin~ (which is the same gnuplot file), with the axes x, y defined as the first and second value in the file (which will be separated by a pipe character), and with the specified titles.
|
||||
|
||||
To accomplish this in the code, we'll add:
|
||||
#+begin_src python
|
||||
line_definitions = ", ".join(
|
||||
[
|
||||
f""""-" using 1:2 title '{name}' with lines"""
|
||||
for container_id, name in sorted(labels.items())
|
||||
]
|
||||
)
|
||||
print("plot", line_definitions)
|
||||
#+end_src
|
||||
|
||||
And then finally we need to output the data for each container, separating each container with an "e". For example, the data for two containers, one that goes from 100 bytes to 120 bytes of memory over the span of 8 seconds, and another that goes from 300 bytes to 900 bytes of memory over the span of 6 seconds, would look like:
|
||||
#+begin_src text
|
||||
0|100
|
||||
2|140
|
||||
4|290
|
||||
6|118
|
||||
8|120
|
||||
e
|
||||
0|300
|
||||
2|450
|
||||
4|650
|
||||
6|900
|
||||
#+end_src
|
||||
|
||||
To accomplish this in code, we loop over the containers, and over the samples for each container, then we subtract the time the container started from the timestamp (which aligns all containers to starting at 0 on the left in the graph, regardless of when they started up) and print out the data:
|
||||
|
||||
#+begin_src python
|
||||
for container_id in sorted(labels.keys()):
|
||||
start_time = int(starting_time_per_container[container_id].timestamp())
|
||||
for sample in sorted(samples, key=lambda x: x.instant):
|
||||
if container_id in sample.stats:
|
||||
print(
|
||||
"|".join(
|
||||
[
|
||||
str(int((sample.instant).timestamp()) - start_time),
|
||||
str(sample.stats[container_id].memory_usage_bytes),
|
||||
]
|
||||
)
|
||||
)
|
||||
print("e")
|
||||
|
||||
#+end_src
|
||||
|
||||
* Run and render
|
||||
That is everything we need. The full script is below, but first to test the script:
|
||||
|
||||
1. Run
|
||||
#+begin_src bash
|
||||
python graph_docker_memory.py | tee memory.gnuplot
|
||||
#+end_src
|
||||
2. Then launch a docker container or two, and perform some actions like install a package. This gives us changing data to create a more interesting line.
|
||||
3. Then close all dokcer containers you have open.
|
||||
|
||||
Now ~memory.gnuplot~ should contain a full gnuplot definition to graph the memory usage of your containers. We can generate an SVG with:
|
||||
#+begin_src bash
|
||||
gnuplot memory.gnuplot > memory.svg
|
||||
#+end_src
|
||||
|
||||
And then open memory.svg in your preferred image viewer and/or web browser to see your graph. It should look like:
|
||||
|
||||
[[./memory.svg]]
|
||||
|
||||
You may notice that the underscore in the container names renders oddly. We just need to escape the underscore when writing the container names. This has been added to the full script below. Also the full script adds support for defining horizontal lines (for example, if you have a memory limit of 1GiB, you might want to put a red horizontal line at 1GiB).
|
||||
|
||||
* Full script
|
||||
Below is the full version of the script in one piece:
|
||||
#+begin_src python
|
||||
#!/usr/bin/env python
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta
|
||||
from time import sleep
|
||||
from typing import Collection, Final, NewType, Tuple
|
||||
|
||||
ContainerId = NewType("ContainerId", str)
|
||||
ContainerName = NewType("ContainerName", str)
|
||||
|
||||
SAMPLE_INTERVAL_SECONDS: Final[int] = 2
|
||||
|
||||
|
||||
@dataclass
|
||||
class Sample:
|
||||
instant: datetime
|
||||
stats: dict[ContainerId, Stats]
|
||||
|
||||
|
||||
@dataclass
|
||||
class Stats:
|
||||
memory_usage_bytes: int
|
||||
|
||||
|
||||
def main():
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
samples: list[Sample] = []
|
||||
labels: dict[ContainerId, ContainerName] = {}
|
||||
first_pass = True
|
||||
# First wait for any docker container to exist.
|
||||
while True:
|
||||
sample, labels_in_sample = take_sample()
|
||||
if labels_in_sample:
|
||||
break
|
||||
if first_pass:
|
||||
first_pass = False
|
||||
logging.info("Waiting for a docker container to exist to start recording.")
|
||||
sleep(1)
|
||||
# And then record memory until no containers exist.
|
||||
while True:
|
||||
sample, labels_in_sample = take_sample()
|
||||
if not labels_in_sample:
|
||||
break
|
||||
samples.append(sample)
|
||||
labels = {**labels, **labels_in_sample}
|
||||
sleep(SAMPLE_INTERVAL_SECONDS)
|
||||
if labels:
|
||||
# Draws a red horizontal line at 32 GiB since that is the memory limit for cloud run.
|
||||
write_plot(
|
||||
samples,
|
||||
labels,
|
||||
horizontal_lines=[(32 * 1024**3, "red", "Cloud Run Max Memory")],
|
||||
)
|
||||
|
||||
|
||||
def write_plot(
|
||||
samples: Collection[Sample],
|
||||
labels: dict[ContainerId, ContainerName],
|
||||
,*,
|
||||
horizontal_lines: Collection[Tuple[int, str, str | None]] = [],
|
||||
):
|
||||
starting_time_per_container = {
|
||||
container_id: min(
|
||||
(sample.instant for sample in samples if container_id in sample.stats)
|
||||
)
|
||||
for container_id in labels.keys()
|
||||
}
|
||||
print(
|
||||
"""set terminal svg background '#FFFFFF'
|
||||
set title 'Docker Memory Usage'
|
||||
set xdata time
|
||||
set timefmt '%s'
|
||||
set format x '%tH:%tM:%tS'
|
||||
# Please note this is in SI units (base 10), not IEC (base 2). So, for example, this would show a Gigabyte, not a Gibibyte.
|
||||
set format y '%.0s%cB'
|
||||
set datafile separator "|"
|
||||
"""
|
||||
)
|
||||
for y_value, color, label in horizontal_lines:
|
||||
print(
|
||||
f'''set arrow from graph 0, first {y_value} to graph 1, first {y_value} nohead linewidth 2 linecolor rgb "{color}"'''
|
||||
)
|
||||
if label is not None:
|
||||
print(f"""set label "{label}" at graph 0, first {y_value} offset 1,-0.5""")
|
||||
|
||||
# Include the horizontal lines in the range
|
||||
if len(horizontal_lines) > 0:
|
||||
print(f"""set yrange [*:{max(x[0] for x in horizontal_lines)}<*]""")
|
||||
line_definitions = ", ".join(
|
||||
[
|
||||
f""""-" using 1:2 title '{gnuplot_escape(name)}' with lines"""
|
||||
for container_id, name in sorted(labels.items())
|
||||
]
|
||||
)
|
||||
print("plot", line_definitions)
|
||||
for container_id in sorted(labels.keys()):
|
||||
start_time = int(starting_time_per_container[container_id].timestamp())
|
||||
for sample in sorted(samples, key=lambda x: x.instant):
|
||||
if container_id in sample.stats:
|
||||
print(
|
||||
"|".join(
|
||||
[
|
||||
str(int((sample.instant).timestamp()) - start_time),
|
||||
str(sample.stats[container_id].memory_usage_bytes),
|
||||
]
|
||||
)
|
||||
)
|
||||
print("e")
|
||||
|
||||
|
||||
def gnuplot_escape(inp: str) -> str:
|
||||
out = ""
|
||||
for c in inp:
|
||||
if c == "_":
|
||||
out += "\\"
|
||||
out += c
|
||||
return out
|
||||
|
||||
|
||||
def take_sample() -> Tuple[Sample, dict[ContainerId, ContainerName]]:
|
||||
labels: dict[ContainerId, ContainerName] = {}
|
||||
stats: dict[ContainerId, Stats] = {}
|
||||
docker_inspect = subprocess.run(
|
||||
["docker", "stats", "--no-stream", "--no-trunc", "--format", "json"],
|
||||
stdout=subprocess.PIPE,
|
||||
)
|
||||
for container_stat in (
|
||||
json.loads(l) for l in docker_inspect.stdout.decode("utf8").splitlines()
|
||||
):
|
||||
if not container_stat["ID"]:
|
||||
# When containers are starting up, they sometimes have no ID and "--" as the name.
|
||||
continue
|
||||
labels[ContainerId(container_stat["ID"])] = ContainerName(
|
||||
container_stat["Name"]
|
||||
)
|
||||
memory_usage = parse_mem_usage(container_stat["MemUsage"])
|
||||
stats[ContainerId(container_stat["ID"])] = Stats(
|
||||
memory_usage_bytes=memory_usage
|
||||
)
|
||||
for container_id, container_stat in stats.items():
|
||||
logging.info(
|
||||
f"Recorded stat {labels[container_id]}: {container_stat.memory_usage_bytes} bytes"
|
||||
)
|
||||
return Sample(instant=datetime.now(), stats=stats), labels
|
||||
|
||||
|
||||
def parse_mem_usage(mem_usage: str) -> int:
|
||||
parsed_mem_usage = re.match(
|
||||
r"(?P<number>[0-9]+\.?[0-9]*)(?P<unit>[^\s]+)", mem_usage
|
||||
)
|
||||
if parsed_mem_usage is None:
|
||||
raise Exception(f"Invalid Mem Usage: {mem_usage}")
|
||||
number = float(parsed_mem_usage.group("number"))
|
||||
unit = parsed_mem_usage.group("unit")
|
||||
for multiplier, identifier in enumerate(["B", "KiB", "MiB", "GiB", "TiB"]):
|
||||
if unit == identifier:
|
||||
return int(number * (1024**multiplier))
|
||||
raise Exception(f"Unrecognized unit: {unit}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
#+end_src
|
||||
26
posts/2024/10/18/graph-docker-memory/memory.gnuplot
Normal file
26
posts/2024/10/18/graph-docker-memory/memory.gnuplot
Normal file
@ -0,0 +1,26 @@
|
||||
set terminal svg background '#FFFFFF'
|
||||
set title 'Docker Memory Usage'
|
||||
set xdata time
|
||||
set timefmt '%s'
|
||||
set format x '%tH:%tM:%tS'
|
||||
# Please note this is in SI units (base 10), not IEC (base 2). So, for example, this would show a Gigabyte, not a Gibibyte.
|
||||
set format y '%.0s%cB'
|
||||
set datafile separator "|"
|
||||
|
||||
plot "-" using 1:2 title 'fervent_sinoussi' with lines, "-" using 1:2 title 'intelligent_visvesvaraya' with lines
|
||||
0|552960
|
||||
5|10371465
|
||||
9|10285481
|
||||
14|10256121
|
||||
18|10256121
|
||||
22|10256121
|
||||
26|10288627
|
||||
29|10288627
|
||||
33|10863247
|
||||
e
|
||||
0|552960
|
||||
5|552960
|
||||
9|897024
|
||||
14|60953722
|
||||
18|21349007
|
||||
e
|
||||
166
posts/2024/10/18/graph-docker-memory/memory.svg
Normal file
166
posts/2024/10/18/graph-docker-memory/memory.svg
Normal file
@ -0,0 +1,166 @@
|
||||
<?xml version="1.0" encoding="utf-8" standalone="no"?>
|
||||
<svg
|
||||
width="600" height="480"
|
||||
viewBox="0 0 600 480"
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||
>
|
||||
|
||||
<title>Gnuplot</title>
|
||||
<desc>Produced by GNUPLOT 6.0 patchlevel 1 </desc>
|
||||
|
||||
<g id="gnuplot_canvas">
|
||||
|
||||
<rect x="0" y="0" width="600" height="480" fill="#ffffff"/>
|
||||
<defs>
|
||||
|
||||
<circle id='gpDot' r='0.5' stroke-width='0.5' stroke='currentColor'/>
|
||||
<path id='gpPt0' stroke-width='0.222' stroke='currentColor' d='M-1,0 h2 M0,-1 v2'/>
|
||||
<path id='gpPt1' stroke-width='0.222' stroke='currentColor' d='M-1,-1 L1,1 M1,-1 L-1,1'/>
|
||||
<path id='gpPt2' stroke-width='0.222' stroke='currentColor' d='M-1,0 L1,0 M0,-1 L0,1 M-1,-1 L1,1 M-1,1 L1,-1'/>
|
||||
<rect id='gpPt3' stroke-width='0.222' stroke='currentColor' x='-1' y='-1' width='2' height='2'/>
|
||||
<rect id='gpPt4' stroke-width='0.222' stroke='currentColor' fill='currentColor' x='-1' y='-1' width='2' height='2'/>
|
||||
<circle id='gpPt5' stroke-width='0.222' stroke='currentColor' cx='0' cy='0' r='1'/>
|
||||
<use xlink:href='#gpPt5' id='gpPt6' fill='currentColor' stroke='none'/>
|
||||
<path id='gpPt7' stroke-width='0.222' stroke='currentColor' d='M0,-1.33 L-1.33,0.67 L1.33,0.67 z'/>
|
||||
<use xlink:href='#gpPt7' id='gpPt8' fill='currentColor' stroke='none'/>
|
||||
<use xlink:href='#gpPt7' id='gpPt9' stroke='currentColor' transform='rotate(180)'/>
|
||||
<use xlink:href='#gpPt9' id='gpPt10' fill='currentColor' stroke='none'/>
|
||||
<use xlink:href='#gpPt3' id='gpPt11' stroke='currentColor' transform='rotate(45)'/>
|
||||
<use xlink:href='#gpPt11' id='gpPt12' fill='currentColor' stroke='none'/>
|
||||
<path id='gpPt13' stroke-width='0.222' stroke='currentColor' d='M0,1.330 L1.265,0.411 L0.782,-1.067 L-0.782,-1.076 L-1.265,0.411 z'/>
|
||||
<use xlink:href='#gpPt13' id='gpPt14' fill='currentColor' stroke='none'/>
|
||||
<filter id='textbox' filterUnits='objectBoundingBox' x='0' y='0' height='1' width='1'>
|
||||
<feFlood flood-color='#FFFFFF' flood-opacity='1' result='bgnd'/>
|
||||
<feComposite in='SourceGraphic' in2='bgnd' operator='atop'/>
|
||||
</filter>
|
||||
<filter id='greybox' filterUnits='objectBoundingBox' x='0' y='0' height='1' width='1'>
|
||||
<feFlood flood-color='lightgrey' flood-opacity='1' result='grey'/>
|
||||
<feComposite in='SourceGraphic' in2='grey' operator='atop'/>
|
||||
</filter>
|
||||
</defs>
|
||||
<g fill="none" color="#FFFFFF" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M54.53,444.00 L63.53,444.00 M574.82,444.00 L565.82,444.00 '/> <g transform="translate(46.14,447.90)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="end">
|
||||
<text><tspan font-family="Arial" >0B</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M54.53,388.29 L63.53,388.29 M574.82,388.29 L565.82,388.29 '/> <g transform="translate(46.14,392.19)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="end">
|
||||
<text><tspan font-family="Arial" >10MB</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M54.53,332.57 L63.53,332.57 M574.82,332.57 L565.82,332.57 '/> <g transform="translate(46.14,336.47)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="end">
|
||||
<text><tspan font-family="Arial" >20MB</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M54.53,276.86 L63.53,276.86 M574.82,276.86 L565.82,276.86 '/> <g transform="translate(46.14,280.76)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="end">
|
||||
<text><tspan font-family="Arial" >30MB</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M54.53,221.15 L63.53,221.15 M574.82,221.15 L565.82,221.15 '/> <g transform="translate(46.14,225.05)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="end">
|
||||
<text><tspan font-family="Arial" >40MB</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M54.53,165.44 L63.53,165.44 M574.82,165.44 L565.82,165.44 '/> <g transform="translate(46.14,169.34)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="end">
|
||||
<text><tspan font-family="Arial" >50MB</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M54.53,109.72 L63.53,109.72 M574.82,109.72 L565.82,109.72 '/> <g transform="translate(46.14,113.62)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="end">
|
||||
<text><tspan font-family="Arial" >60MB</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M54.53,54.01 L63.53,54.01 M574.82,54.01 L565.82,54.01 '/> <g transform="translate(46.14,57.91)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="end">
|
||||
<text><tspan font-family="Arial" >70MB</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M54.53,444.00 L54.53,435.00 M54.53,54.01 L54.53,63.01 '/> <g transform="translate(54.53,465.90)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="middle">
|
||||
<text><tspan font-family="Arial" >0:00:00</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M128.86,444.00 L128.86,435.00 M128.86,54.01 L128.86,63.01 '/> <g transform="translate(128.86,465.90)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="middle">
|
||||
<text><tspan font-family="Arial" >0:00:05</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M203.18,444.00 L203.18,435.00 M203.18,54.01 L203.18,63.01 '/> <g transform="translate(203.18,465.90)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="middle">
|
||||
<text><tspan font-family="Arial" >0:00:10</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M277.51,444.00 L277.51,435.00 M277.51,54.01 L277.51,63.01 '/> <g transform="translate(277.51,465.90)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="middle">
|
||||
<text><tspan font-family="Arial" >0:00:15</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M351.84,444.00 L351.84,435.00 M351.84,54.01 L351.84,63.01 '/> <g transform="translate(351.84,465.90)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="middle">
|
||||
<text><tspan font-family="Arial" >0:00:20</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M426.17,444.00 L426.17,435.00 M426.17,54.01 L426.17,63.01 '/> <g transform="translate(426.17,465.90)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="middle">
|
||||
<text><tspan font-family="Arial" >0:00:25</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M500.49,444.00 L500.49,435.00 M500.49,54.01 L500.49,63.01 '/> <g transform="translate(500.49,465.90)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="middle">
|
||||
<text><tspan font-family="Arial" >0:00:30</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M574.82,444.00 L574.82,435.00 M574.82,54.01 L574.82,63.01 '/> <g transform="translate(574.82,465.90)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="middle">
|
||||
<text><tspan font-family="Arial" >0:00:35</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M54.53,54.01 L54.53,444.00 L574.82,444.00 L574.82,54.01 L54.53,54.01 Z '/></g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
</g>
|
||||
<g id="gnuplot_plot_1" ><title>ferventsinoussi</title>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<g transform="translate(507.09,75.91)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="end">
|
||||
<text><tspan font-family="Arial" >fervent</tspan><tspan font-family="Arial" font-size="9.6" dy="3.60px">s</tspan><tspan font-family="Arial" font-size="12.0" dy="-3.60px">inoussi</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='rgb(148, 0, 211)' d='M515.48,72.01 L558.04,72.01 M54.53,440.92 L128.86,386.22 L188.32,386.70 L262.65,386.86 L322.11,386.86 L381.57,386.86
|
||||
L441.03,386.68 L485.63,386.68 L545.09,383.48 '/></g>
|
||||
</g>
|
||||
<g id="gnuplot_plot_2" ><title>intelligentvisvesvaraya</title>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<g transform="translate(507.09,93.91)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="end">
|
||||
<text><tspan font-family="Arial" >intelligent</tspan><tspan font-family="Arial" font-size="9.6" dy="3.60px">v</tspan><tspan font-family="Arial" font-size="12.0" dy="-3.60px">isvesvaraya</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='rgb( 0, 158, 115)' d='M515.48,90.01 L558.04,90.01 M54.53,440.92 L128.86,440.92 L188.32,439.00 L262.65,104.41 L322.11,325.06 '/></g>
|
||||
</g>
|
||||
<g fill="none" color="#FFFFFF" stroke="rgb( 0, 158, 115)" stroke-width="2.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="black" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<path stroke='black' d='M54.53,54.01 L54.53,444.00 L574.82,444.00 L574.82,54.01 L54.53,54.01 Z '/></g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
<g transform="translate(314.67,30.91)" stroke="none" fill="black" font-family="Arial" font-size="12.00" text-anchor="middle">
|
||||
<text><tspan font-family="Arial" >Docker Memory Usage</tspan></text>
|
||||
</g>
|
||||
</g>
|
||||
<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="butt" stroke-linejoin="miter">
|
||||
</g>
|
||||
</g>
|
||||
</svg>
|
||||
|
||||
|
After Width: | Height: | Size: 12 KiB |
Loading…
Reference in New Issue
Block a user