[{"content":"Green coffee contains sucrose, glucose and fructose; the roasted bean and the cup contain almost none. Black coffee with no sugar added has no meaningful sugar, yet it can taste distinctly sweet.\nWhere the sugar goes: roasting drives the Maillard reaction and caramelisation, which consume sugars and convert them into hundreds of aromatic compounds. What is left is a trace of sugar-derived molecules — and with bitterness and acidity reduced around them, the brain reads those as sweetness.\nSame effect as bread: bread needs no added sugar to develop aromas that taste toasted, caramelised or sweet; the crust produces them.\nThe distinction worth keeping: a naturally sweet coffee is not a coffee with sugar in it. When a bag describes caramel, chocolate, honey or fruit, none of those ingredients are present — they are what the bean and the roast produced. That vocabulary describes aroma, not a sweetener.\nWhich is why sweetness is a quality signal: it comes from a ripe bean, a controlled roast, an extraction that stops before bitterness arrives. Off-flavours and harsh bitterness mask it.\nReferences Specialty Coffee Association: flavour wheel ","date":"2026-08-23T00:00:00Z","dateFormatted":"23 Aug 2026","permalink":"https://genezini.com/garden/coffee-sweetness/","title":"Why coffee tastes sweet without sugar"},{"content":"A \u0026ldquo;multi-state\u0026rdquo; USB WiFi adapter carries on-board storage with a Windows driver, so on first plug it enumerates as a CD-ROM or flash drive. Windows autoruns the installer; other OSes see only a disk and no network interface, so the adapter must be told to switch.\nMost mainstream distributions — Ubuntu, Fedora, Debian, OpenSUSE, Arch — ship usb-modeswitch with the adapter in its data files and switch silently; hand-switch only when the package is missing, the data predates the adapter, or the device is a rebadged clone.\nFinding out lsusb sudo apt install usb-modeswitch usb-modeswitch-data sudo usb_modeswitch -K -W -v 0e8d -p 2870 # vendor:product from lsusb -K sends the eject, -W removes the fake storage. If lsusb then shows a new product ID and ip link a wireless interface, the switch worked — make it permanent.\nMaking it permanent Two files: a udev rule triggering the switch on plug, and a config file for the device. The rule goes in /lib/udev/rules.d/40-usb_modeswitch.rules, below the GOTO=\u0026quot;modeswitch_rules_end\u0026quot; line near the top:\n# COMFAST CF-WU782AC/WU785AC, TEROW ROW02FD ATTR{idVendor}==\u0026#34;0e8d\u0026#34;, ATTR{idProduct}==\u0026#34;2870\u0026#34;, RUN+=\u0026#34;usb_modeswitch \u0026#39;/%k\u0026#39;\u0026#34; The config file is named after the original IDs, /usr/share/usb_modeswitch/0e8d:2870:\n1 2 3 TargetVendor=0x0e8d TargetProductList=\u0026#34;7612\u0026#34; StandardEject=1 TargetVendor/TargetProductList are the IDs the device takes after switching — what the kernel binds a driver to; StandardEject=1 is the common case, and quirky devices need a MessageContent byte sequence instead.\nReboot, or replug the adapter, and it should come up as WiFi.\nAdapter Original IDs Target product Config file COMFAST CF-WU782AC, CF-WU785AC, TEROW ROW02FD 0e8d:2870 7612 /usr/share/usb_modeswitch/0e8d:2870 Techkey 5B12/5B10 (Realtek 8811cu) 0bda:1a2b — rule only, inline -K -v 0bda -p 1a2b D-Link DWA-X1850, Alfa AWUS036AXER 0bda:1a2b 8852 /usr/share/usb_modeswitch/0bda:1a2b The 8811cu entry is the exception: some adapters switch with just the -K eject in the rule, no config file.\nATTR{idVendor}==\u0026#34;0bda\u0026#34;, ATTR{idProduct}==\u0026#34;1a2b\u0026#34;, RUN+=\u0026#34;/usr/sbin/usb_modeswitch -K -v 0bda -p 1a2b\u0026#34; Two rule forms: RUN+=\u0026quot;usb_modeswitch '/%k'\u0026quot; reads the config file on every plug; an inline usb_modeswitch -K … hardcodes the command. Prefer the config file — a switched device that re-enumerates back to storage usually means a rule that never ran.\nTurning it off To stop it fighting over a device, edit /etc/usb_modeswitch.conf:\nDisableSwitching=1 Then reboot. Disabling is global, useful when something else handles the modem or dongle.\nReferences morrownr/USB-WiFi: How to Modeswitch usb_modeswitch ","date":"2026-06-07T00:00:00Z","dateFormatted":"07 Jun 2026","permalink":"https://genezini.com/garden/usb-modeswitch/","title":"USB WiFi adapters that show up as a CD-ROM"},{"content":"An analytical database embedded like SQLite — no server, one file or in memory — but built for scans, aggregations and joins, not transactions: SQLite for analytics.\nWhere SQLite stores a row together, DuckDB stores each column separately:\nrow-oriented: [id=1, name=Alice, age=30] [id=2, name=Bob, age=25] column-oriented: id: [1,2,3] name: [Alice,Bob,Carol] age: [30,25,40] SELECT AVG(age) FROM users then reads one column, not every column of every row. It also executes vectorized — chunks of around 1024 rows, not one row at a time — using the CPU cache better and allowing SIMD.\nDuckDB SQLite Layout columnar row Built for OLAP OLTP Aggregations at scale excellent adequate Embedded, single file yes yes What it is good at Querying files without importing:\n1 2 3 SELECT * FROM \u0026#39;sales.csv\u0026#39;; SELECT COUNT(*) FROM read_csv(\u0026#39;sales.csv\u0026#39;); SELECT * FROM \u0026#39;data.parquet\u0026#39;; Parquet is the strong case — often without loading the whole dataset into memory, making DuckDB a local engine over data lakes and Iceberg, Delta Lake, Hive and Parquet layouts.\nThe SQL is analytical: GROUP BY aggregations, joins, window functions such as ROW_NUMBER() OVER (PARTITION BY country ORDER BY score DESC), and CREATE TABLE clean_data AS SELECT … for local ETL. In Python it reads Pandas, Polars and Arrow data directly:\n1 2 import duckdb duckdb.sql(\u0026#34;SELECT * FROM my_dataframe WHERE value \u0026gt; 100\u0026#34;) Throughput: hundreds of millions of rows on a laptop is realistic for scans, filters, joins and aggregations — near much larger systems.\nWhat it is not Not a web application database: user accounts, carts, banking, high-concurrency OLTP belong in PostgreSQL, MySQL or MariaDB. Writes are supported but not the design centre; many simultaneous writers are outside its remit.\nNot a vector database: array types, embedding storage and similarity-search extensions exist but are peripheral next to Milvus, Qdrant or Weaviate. Production retrieval with millions of embeddings wants a dedicated vector store; the two are complementary.\nThe common split: SQLite for application data, DuckDB for reporting, log analysis and Parquet.\n","date":"2026-06-06T00:00:00Z","dateFormatted":"06 Jun 2026","permalink":"https://genezini.com/garden/duckdb/","title":"DuckDB: SQLite for analytics, not a vector database"},{"content":"The foreground job Ctrl+Z sends SIGTSTP to the foreground job: frozen, memory and file descriptors kept open.\njobs # list suspended jobs and their numbers fg # resume the most recent in the foreground bg # resume it in the background fg %2 # resume job 2 bg %2 Anything else Find the PID, then signal:\nps aux | grep \u0026lt;process_name\u0026gt; pidof \u0026lt;process_name\u0026gt; sudo kill -STOP \u0026lt;PID\u0026gt; # or kill -SIGSTOP sudo kill -CONT \u0026lt;PID\u0026gt; # or kill -SIGCONT SIGSTOP cannot be caught or ignored — reliable when you do not control the process.\nIn a container Host signals do not reach a container\u0026rsquo;s process; use the runtime:\ndocker pause \u0026lt;container_id\u0026gt; docker unpause \u0026lt;container_id\u0026gt; A paused process uses no CPU but keeps its memory. SIGSTOP state is not persisted, so a suspended terminal job dies at reboot; anything that must hold across restarts needs hibernation or explicit state saving.\nReferences signal(7) ","date":"2026-05-30T00:00:00Z","dateFormatted":"30 May 2026","permalink":"https://genezini.com/garden/pause-process/","title":"Pausing and resuming a process"},{"content":"Three steps in sys mode — edit the repositories file, refresh the index, upgrade everything; in-branch security updates are separate. Back up and read the release notes first.\nPoint at the new branch setup-apkrepos e opens it in vi; or in one line, moving 3.22 to 3.23:\nsed -i -e \u0026#39;s/ v3.22 / v3.23 /g\u0026#39; /etc/apk/repositories http://dl-3.alpinelinux.org/alpine/v3.22/main http://dl-3.alpinelinux.org/alpine/v3.22/community latest-stable avoids editing the file each release:\nhttp://dl-3.alpinelinux.org/alpine/latest-stable/main http://dl-3.alpinelinux.org/alpine/latest-stable/community ⚠️ latest-stable also means an unexpected release upgrade can begin on its own — the trade for never editing the file again.\nUpgrade apk update apk add --upgrade apk-tools apk upgrade --available Upgrading apk-tools first costs nothing and is sometimes required; --available upgrades every package even at matching versions, as musl changes occasionally require.\n-U / --update-cache equals apk update just before it, but refreshes only a stale index; apk upgrade -U is shorthand.\nRestart services after upgrade; a kernel upgrade needs a reboot:\nsync reboot Moving to edge Same procedure, every entry on edge:\n#/media/cdrom/apks http://dl-cdn.alpinelinux.org/alpine/edge/main http://dl-cdn.alpinelinux.org/alpine/edge/community @testing http://dl-cdn.alpinelinux.org/alpine/edge/testing Downgrading from edge is unsupported: the documented order is apk fix, apk update, apk upgrade, apk upgrade -a, apk upgrade -al; a package may then need removing and reinstalling by hand.\nReferences Alpine Wiki: Upgrading to a new release branch Alpine Wiki: package management ","date":"2026-05-30T00:00:00Z","dateFormatted":"30 May 2026","permalink":"https://genezini.com/garden/alpine-upgrade/","title":"Upgrading Alpine to a new release branch"},{"content":"Atuin replaces shell history search: full-text and fuzzy search over a SQLite database rather than a flat history file, per-directory and per-session context, statistics, optional end-to-end encryption, and sync between machines. In zsh it replaces the default Ctrl+R reverse search.\nSetup eval \u0026#34;$(atuin init zsh)\u0026#34; Near the end of .zshrc, then exec zsh.\nfzf fzf is optional (Atuin has its own fuzzy search) but still covers file finding, git branch switching, process selection, piped input. Atuin for history, fzf for everything else.\nBoth want Ctrl+R; last initialised wins. For Atuin:\nsource \u0026lt;(fzf --zsh) eval \u0026#34;$(atuin init zsh)\u0026#34; For fzf: eval \u0026quot;$(atuin init zsh --disable-up-arrow)\u0026quot;, remap by hand.\nOther commands:\natuin import auto # pull in existing zsh history atuin stats atuin search docker atuin history list atuin status # sync enabled? logged in? which address? Sync Sync needs an account even self-hosted:\natuin register -u USER -e you@example.com --sync-address https://atuin.example.com atuin login -u USER -p PASSWORD --sync-address https://atuin.example.com atuin sync sync_address in ~/.config/atuin/config.toml must be the bare origin — no /api, no /api/v0, no trailing slash. Without a login, history stays local, the server database empty.\nChecking that sync works The server logs almost nothing when healthy, nothing at INFO; verify from the client and database:\natuin sync -f # prints uploaded / downloaded record counts ATUIN_LOG=debug atuin sync # requests to /api/v0/record and /sync atuin status Probe the API directly; inside the container it listens on 8888:\ncurl -i https://atuin.example.com/api/v0/ping curl -i https://atuin.example.com/api/v0/store/history docker exec -it atuin sh wget -S -O - http://localhost:8888/api/v0/ping 401 is good — responded, wants auth. 404 means the route does not exist, usually a reverse-proxy problem. A root-level 404 on / proves nothing; that path is not served.\nProxy mistake that strips the path:\nlocation /atuin/ { proxy_pass http://atuin:8888/; # breaks the API routes } Serve at the root or a dedicated subdomain; answers inside the container but not outside mean the proxy, not Atuin.\nConfirm data lands (Postgres):\nSELECT count(*) FROM history; SELECT count(*) FROM records; SELECT username FROM users; End to end: echo \u0026quot;test-atuin-sync\u0026quot; on one machine, atuin sync on both, then atuin search test-atuin-sync.\nIf the client logs nothing, raise server verbosity:\nenvironment: RUST_LOG: debug then docker compose up -d --force-recreate. Usual causes of silence: the client never attempts a sync, sync_address is wrong, HTTPS or the reverse proxy blocks it, or the container runs without debug logging.\nReferences Atuin Atuin server docs ","date":"2026-05-24T00:00:00Z","dateFormatted":"24 May 2026","permalink":"https://genezini.com/garden/atuin/","title":"Atuin: shell history that syncs"},{"content":"fd is a filesystem walker, a modern find replacement: scans on demand, respects .gitignore by default, takes regex or glob patterns, executes per result.\nfd config ~/.config fd . ~/projects | fzf plocate is a database-backed filename indexer, compatible with locate: queries a prebuilt index, never touches the disk.\nplocate firefox Trade-off: freshness for speed. fd sees the disk now — current, slow on large trees; plocate is near-instant but knows only what updatedb last indexed, usually via a systemd timer or cron.\nfd plocate Source live filesystem prebuilt index Speed slower, scales with tree size near-instant Freshness always current stale until updatedb Pipeline-friendly yes weaker Scope a directory you name whole system Pick by question: plocate nvim to find where something lives on the machine, then fd init.lua ~/.config/nvim to search from there.\nEasy to confuse: find is the classic recursive walk; ripgrep searches file contents; fzf is an interactive filter over any list; mlocate is the older locate implementation.\nReferences fd plocate ","date":"2026-05-24T00:00:00Z","dateFormatted":"24 May 2026","permalink":"https://genezini.com/garden/fd-vs-plocate/","title":"fd and plocate: live search versus an index"},{"content":"The factory Analytics turns raw events — a click, an add-to-cart, a purchase, an approval — into trustworthy, queryable data:\nRaw logs → Cleaned data → Business metrics → Dashboards Databricks layer names, same progression:\nLayer Holds Shape Bronze raw files, as received append-only, untyped Silver cleaned, joined, deduplicated typed tables Gold business metrics aggregated, query-ready CSV is the starting interchange format; Parquet is where it settles — columnar, compressed, typed, readable by DuckDB, Spark and Trino. A pipeline\u0026rsquo;s job is movement between layers; the tool is whichever engine does that without becoming maintenance.\nGrouping text without a model Finding related chunks needs no LLM, only a similarity function over vectors:\ncosine similarity = (A · B) / (‖A‖ ‖B‖) Embedding each chunk with sentence-transformers and comparing vectors is fast, local, no API calls, good enough semantically; all-MiniLM-L6-v2 is the fast option, all-mpnet-base-v2 the better, slower one.\n1 2 from sentence_transformers import SentenceTransformer from sklearn.metrics.pairwise import cosine_similarity An LLM is optional, for writing the result up only.\nHDBSCAN Hierarchical Density-Based Spatial Clustering of Applications with Noise — DBSCAN improved: varying densities, a cluster hierarchy rather than one flat partition, the best level chosen automatically, outliers marked as noise rather than forced into a cluster.\nCore distance — distance to the k-th nearest neighbour, capturing local density: small in dense regions, large in sparse ones.\nMutual reachability distance — not raw distance between two points:\nmax(core_dist(a), core_dist(b), distance(a, b)) The maximum stops two points in sparse regions being called close merely for proximity; both must sit in regions dense by their own standards, so loose points cannot form a cluster that looks real.\nFor text: cluster embedded chunks, let HDBSCAN label the rest as noise rather than inventing topics, then ask a model to name the clusters.\n","date":"2026-05-08T00:00:00Z","dateFormatted":"08 May 2026","permalink":"https://genezini.com/garden/analytics-basics/","title":"Analytics basics: layers, parquet, similarity, clustering"},{"content":"git config --global core.excludesfile ~/.gitignore_global touch ~/.gitignore_global Ignored in every repository on the machine: things about your environment, not the project:\n.DS_Store # macOS Thumbs.db # Windows .vscode/ # VS Code .idea/ # JetBrains *.log .env A global ignore is personal, machine-wide, shared with no one. A repository\u0026rsquo;s .gitignore is committed and shared, describing what the project never wants tracked. Editor and OS junk goes global; project tooling\u0026rsquo;s output goes in the repo.\n","date":"2026-05-04T00:00:00Z","dateFormatted":"04 May 2026","permalink":"https://genezini.com/garden/global-gitignore/","title":"A global gitignore for your own machine"},{"content":"ss is the modern default, usually installed:\nsudo ss -lptn \u0026#39;sport = :3000\u0026#39; LISTEN 0 128 *:3000 *:* users:((\u0026#34;node\u0026#34;,pid=12345,fd=18)) lsof gives a clearer PID plus command path:\nsudo lsof -i :3000 netstat on older systems, from net-tools:\nsudo netstat -tulpn | grep :3000 Killing the holder — a plain kill first, -9 only if it refuses:\nsudo kill 12345 sudo fuser -k 3000/tcp To check a port is open, ignoring the owner:\nss -tuln | grep :3000 Usual cases: a taken port, or a Docker container colliding with a local process; in containers the answer comes from docker ps or the orchestrator, not ss on the host.\nReferences ss(8) lsof(8) ","date":"2026-05-04T00:00:00Z","dateFormatted":"04 May 2026","permalink":"https://genezini.com/garden/find-process-by-port/","title":"Find which process holds a port"},{"content":"Setting a variable without logging out A modern GNOME session runs under systemd; GUI applications inherit that environment:\nsystemctl --user set-environment MY_VAR=value dbus-update-activation-environment --systemd MY_VAR=value The first sets it for systemd user services and later shell-launched applications; the second makes D-Bus-activated apps see it, how most GNOME apps start.\necho $MY_VAR prints nothing: variables are copied into a process at start, and your shell predates the change. Shell and systemd user session do not propagate:\nsystemctl --user show-environment | grep MY_VAR eval \u0026#34;$(systemctl --user show-environment | grep MY_VAR)\u0026#34; The first confirms the setting, the second imports it into the current shell.\nPersistent, in a drop-in read at session start:\necho \u0026#39;MY_VAR=value\u0026#39; \u0026gt;\u0026gt; ~/.config/environment.d/myenv.conf systemctl --user import-environment MY_VAR dbus-update-activation-environment --systemd MY_VAR Where not to put it: ~/.profile and ~/.bashrc need a new login, /etc/environment a logout or reboot, and /etc/profile reaches shells only, never the GNOME session.\nRunning something at login Autostart, for a GUI program or one needing the session environment — DISPLAY, DBUS_SESSION_BUS_ADDRESS:\nmkdir -p ~/.config/autostart [Desktop Entry] Type=Application Exec=/path/to/your/script.sh Hidden=false NoDisplay=false X-GNOME-Autostart-enabled=true Name=My Login Script Comment=Run my script at GNOME login chmod +x /path/to/your/script.sh A systemd user service, for background, restartable work — more reliable than autostart for anything long-lived:\n[Unit] Description=Run my script at GNOME login After=graphical-session.target [Service] ExecStart=/path/to/your/script.sh Type=simple [Install] WantedBy=default.target systemctl --user daemon-reload systemctl --user enable --now myscript.service A shell RC file, for a text login before GNOME: ~/.bash_profile or ~/.bash_login. Anything needing the GNOME session will not work.\nAnything requiring sudo will not run at login without passwordless sudo — rewrite it as a system service.\nReferences systemd: user environment Desktop Application Autostart Specification ","date":"2026-05-04T00:00:00Z","dateFormatted":"04 May 2026","permalink":"https://genezini.com/garden/gnome-autostart-and-environment/","title":"GNOME environment variables and login scripts"},{"content":"Rows with no match Both find rows in TableA with no TableB counterpart:\n1 2 3 4 5 6 7 8 SELECT a.* FROM TableA a LEFT JOIN TableB b ON b.AId = a.Id WHERE b.AId IS NULL; SELECT a.* FROM TableA a WHERE NOT EXISTS (SELECT 1 FROM TableB b WHERE b.AId = a.Id); NOT EXISTS is the better default:\nCorrectness with NULLs. LEFT JOIN breaks when the join condition is not exactly equivalent, with nullable columns, or when a joined-table filter lands in WHERE instead of ON — silently an inner join. Keep every joined-table filter inside ON:\n1 2 3 4 LEFT JOIN TableB b ON b.AId = a.Id AND b.IsActive = 1 WHERE b.AId IS NULL; Optimizer intent. SQL Server recognizes NOT EXISTS as an anti-semi join, stopping at the first match and using indexes better; modern versions optimize both, but the subquery states intent more clearly and large-table plans follow.\nNOT IN is to be avoided unless the subquery provably cannot return NULLs:\n1 WHERE a.Id NOT IN (SELECT b.AId FROM TableB b) One NULL there and the query returns no rows at all — looks like data loss, not a bug.\nPattern Correct with NULLs Optimizer-friendly Verdict NOT EXISTS yes yes best default LEFT JOIN … IS NULL sometimes usually with care NOT IN no sometimes avoid ","date":"2026-05-04T00:00:00Z","dateFormatted":"04 May 2026","permalink":"https://genezini.com/garden/anti-joins-and-duplicates/","title":"SQL Anti-joins"},{"content":"~/.local/share/applications follows symlinks: a .desktop file can live in dotfiles and be linked in:\nln -s ~/dotfiles/desktop/myapp.desktop ~/.local/share/applications/myapp.desktop xdg-desktop-menu resolves the link, update-desktop-database indexes it, gtk-launch, rofi and dmenu find it. Mode 644, no execute bit:\nchmod 644 myapp.desktop update-desktop-database ~/.local/share/applications gtk-launch myapp A broken symlink makes the launcher vanish silently; Exec= paths resolve literally, not via the symlink.\nExec and $HOME Exec=$HOME/scripts/open-browser.sh %u Works on GNOME, KDE and XFCE, but the Desktop Entry specification does not guarantee it — a stricter launcher may not expand the variable. An absolute path always works:\nExec=/home/daniel/scripts/open-browser.sh %u GNOME on Fedora rejects the bare form: $ is reserved, needs quoting and escaping, and field codes may not sit inside a quoted argument. Portable workaround — %u as an unquoted positional parameter:\nExec=sh -c \u0026#39;exec \u0026#34;$HOME/scripts/open-browser.sh\u0026#34; \u0026#34;$1\u0026#34;\u0026#39; sh %u %u is a single URL, %U accepts several, and a script taking no URL should have neither.\nThe window that will not come forward sh -c breaks GNOME\u0026rsquo;s window association: the shell looks like the launched application, so the window belongs to a process GNOME cannot match — no focus, no icon grouping, may open in the background.\nStartupWMClass fixes it. Run the application and inspect its window:\nwmctrl -lx xprop | grep WM_CLASS Take the second, case-sensitive value:\n[Desktop Entry] Name=My App Exec=sh -c \u0026#39;exec \u0026#34;$HOME/scripts/open-browser.sh\u0026#34; \u0026#34;$1\u0026#34;\u0026#39; sh %u Type=Application StartupWMClass=firefox Cleaner: don\u0026rsquo;t wrap it. A script ending in exec firefox \u0026quot;$@\u0026quot; replaces its own process, so the launcher points straight at it and GNOME sees the real application:\n#!/usr/bin/env bash exec firefox \u0026#34;$@\u0026#34; Exec=/home/daniel/scripts/open-browser.sh %u StartupWMClass=firefox Neither Terminal=false nor StartupNotify=true addresses the cause; nohup, a trailing \u0026amp; or disown make it worse. Validate and watch a launch:\ndesktop-file-validate ~/.local/share/applications/myapp.desktop journalctl --user -f | grep -i -E \u0026#39;gnome|shell|desktop|gio|gtk\u0026#39; ","date":"2026-05-04T00:00:00Z","dateFormatted":"04 May 2026","permalink":"https://genezini.com/garden/desktop-files-and-symlinks/","title":"Symlinked .desktop files, and why the window opens behind"},{"content":"HISTSIZE is interactive-only, so .zshenv is wrong — it runs for every shell, scripts included. It belongs in .zshrc; several files can set the value you see.\ngrep HISTSIZE ~/.zshrc grep HISTSIZE ~/.oh-my-zsh/lib/history.zsh grep HISTSIZE /etc/zsh* Oh My Zsh keeps history settings in ~/.oh-my-zsh/lib/history.zsh, defaults above the 1000 that usually prompts the question; system-wide values come from /etc/zshrc, /etc/zprofile or /etc/zshenv.\nValue and origin:\necho $HISTSIZE echo $SAVEHIST zsh -xv # every file sourced, in order Change it in .zshrc, then source ~/.zshrc:\nHISTSIZE=10000 SAVEHIST=10000 compinstall filename style zstyle \u0026#39;:compinstall\u0026#39; filename \u0026#39;~/.zshrc\u0026#39; The interactive text setup for zsh completion: tells compinstall where to write results; without it compinstall asks each time, with it fixed and dotfiles-friendly.\nIt does nothing alone — only if compinstall is run, and writes completion configuration only: autoload -Uz compinit, compinit, completion zstyle rules and related options. History, aliases, keybindings and PATH are untouched.\nKeep completion configuration out of .zshrc: point the style elsewhere and source the result:\nzstyle \u0026#39;:compinstall\u0026#39; filename \u0026#39;~/.zshrc.compinstall\u0026#39; [[ -f ~/.zshrc.compinstall ]] \u0026amp;\u0026amp; source ~/.zshrc.compinstall zstyle -L \u0026#39;:compinstall\u0026#39; # where the setup would be written ","date":"2026-05-04T00:00:00Z","dateFormatted":"04 May 2026","permalink":"https://genezini.com/garden/zsh-history-settings/","title":"Where HISTSIZE actually comes from"},{"content":"ZRAM compresses memory pages in RAM instead of swapping to disk; the algorithm sets the trade between CPU time and memory saved.\nThe default on most modern distributions is lzo-rle: very fast, low CPU overhead, reasonable ratio. Confirm what is active — the bracketed entry is in use:\ncat /sys/block/zram0/comp_algorithm lzo [lzo-rle] lz4 lz4hc zstd Algorithm CPU cost Speed Ratio lz4 lowest very fast worst lzo-rle low very fast medium lzo low fast medium zstd higher slower best lz4hc high slow better than lz4 Pick lz4 when the CPU is weak or loaded and RAM is cheaper than cycles, lzo-rle for the default balance, zstd when memory is scarce and CPU is not.\nChanging it at runtime:\necho lz4 | sudo tee /sys/block/zram0/comp_algorithm Persistently, with systemd-zram-generator:\n[zram0] zram-size = ram / 2 compression-algorithm = lz4 Config lives at /etc/systemd/zram-generator.conf; the change needs a reboot. A poor ratio costs RAM — pages wait longer to be reclaimed — so lz4 suits a machine already swapping heavily.\nReferences systemd-zram-generator ","date":"2026-05-04T00:00:00Z","dateFormatted":"04 May 2026","permalink":"https://genezini.com/garden/zram-compression/","title":"ZRAM compression: lzo-rle by default, lz4 for CPU"},{"content":"Completion is function-based: on Tab zsh finds a completion function matching the command name, and it must exist. Zsh reads no --help.\nMinimum setup:\nautoload -Uz compinit compinit zstyle \u0026#39;:completion:*\u0026#39; menu select zstyle \u0026#39;:completion:*\u0026#39; matcher-list \u0026#39;m:{a-z}={A-Z}\u0026#39; setopt AUTO_MENU setopt COMPLETE_IN_WORD compinit loads completion and builds a dump file, ~/.zcompdump. Rebuild it if changes do not appear, or zsh warns about an insecure completion directory:\nrm -f ~/.zcompdump* compinit Git completion Git support is zsh\u0026rsquo;s own _git function: once compinit has run, these complete on Tab with refs, remotes and branches:\ngit checkout \u0026lt;TAB\u0026gt; git switch \u0026lt;TAB\u0026gt; git merge \u0026lt;TAB\u0026gt; git rebase \u0026lt;TAB\u0026gt; git push origin \u0026lt;TAB\u0026gt; Oh My Zsh enables it; not required.\nWho ships the script A tool\u0026rsquo;s completion file is named after the command — _myapp — on $fpath, usually ~/.zsh/completions/ or the package manager\u0026rsquo;s site-functions directory. Modern CLIs generate it:\nmyapp completion zsh \u0026gt; ~/.zsh/completions/_myapp compinit Cobra (Go), clap (Rust), yargs or oclif (Node) emit one; writing your own is the fallback for internal tools.\nCompletion for an alias Shell alias for git — alias g=git — must reuse git\u0026rsquo;s completion:\nalias g=git compdef g=git Keep completealiases off, or zsh expands the alias before completion runs and the binding never matches:\nunsetopt completealiases Git\u0026rsquo;s own aliases — co = checkout — are handled by the official script via git help --aliases-for-completion.\nGit aliases that shell out — starting with ! — are opaque shell commands; attach completion to a shell function:\nwa() { command git wa \u0026#34;$@\u0026#34;; } compdef _git-checkout wa Branch completion in your own command autoload -Uz compinit compinit _mycmd() { local -a branches if git rev-parse --is-inside-work-tree \u0026gt;/dev/null 2\u0026gt;\u0026amp;1; then branches=(${(f)\u0026#34;$(git for-each-ref --format=\u0026#39;%(refname:short)\u0026#39; refs/heads 2\u0026gt;/dev/null)\u0026#34;}) _arguments \u0026#39;1:branch:compadd -a branches\u0026#39; else _message \u0026#39;not in a git repository\u0026#39; fi } compdef _mycmd mycmd Add refs/remotes to for-each-ref for remote branches; scope to one subcommand via $words[2]. A branch argument followed by more:\n_arguments \\ \u0026#39;1:branch:_git-branch\u0026#39; \\ \u0026#39;*::args:_files\u0026#39; Helpers by type: _git-branch/_git-switch refs, _files paths, _git-remote remotes.\ncompdef does nothing until compinit has run.\nReferences zsh: completion system zsh: compinit ","date":"2026-05-04T00:00:00Z","dateFormatted":"04 May 2026","permalink":"https://genezini.com/garden/zsh-completion/","title":"Zsh completion: compinit, compdef, and custom commands"},{"content":"Each line was found once and would otherwise be found again; nothing is distribution-specific but the package manager.\nFirmware sudo pacman -S gnome-firmware The libvirt default network libvirt ships the default network inactive: NAT, backed by dnsmasq:\nsudo pacman -S dnsmasq sudo virsh net-list --all sudo virsh net-autostart default sudo virsh net-start default net-autostart survives a reboot; net-start covers only this session.\nFlameshot on Wayland Under Wayland, Flameshot cannot grab the screen from a GNOME shortcut: no terminal to attach to. Wrapping it in script supplies one:\nscript --command \u0026#34;QT_QPA_PLATFORM=wayland flameshot gui\u0026#34; /dev/null Point the custom shortcut at that, not flameshot gui. See Flameshot — Wayland help.\nLow battery warning UPower thresholds live in /etc/UPower/UPower.conf. After editing:\nsudo systemctl restart upower Emergency shell from GRUB When the system no longer reaches a login: at the GRUB menu, highlight the entry, press e, append to the kernel command line:\ninit=/bin/bash Boot with Ctrl+X. Root comes up read-only: remount before editing — /etc/fstab is the usual reason to be here:\nmount -o remount,rw / Fingerprint reader Packages first; the AUR build covers chips stock libfprint does not (a CS9711):\nsudo pacman -S polkit-gnome fprintd yay -S libfprint-CS9711-git Then PAM. In /etc/pam.d/sudo the first line keeps sudo usable from a real terminal without enrolled fingerprints, not a password fall-through; the second accepts the reader:\nauth [success=1 default=ignore] pam_succeed_if.so service in sudo:su:su-l tty in :unknown auth sufficient pam_fprintd.so Polkit needs the same line in /usr/lib/pam.d/polkit-1, or GNOME dialogs ignore it:\nauth sufficient pam_fprintd.so References Arch Wiki: libvirt Arch Wiki: fprint ","date":"2026-05-02T00:00:00Z","dateFormatted":"02 May 2026","permalink":"https://genezini.com/garden/arch-setup-notes/","title":"Arch setup notes: firmware, libvirt networking, fingerprint login"},{"content":"Restoring borg list /path/to/repo borg list /path/to/repo::daily-2026-01-21 home/daniel borg extract /path/to/repo::daily-2026-01-21 home/daniel/Documents extract writes into the current directory and recreates the stored path — from /tmp/restore you get /tmp/restore/home/daniel/Documents, not an overwritten live tree. Listing a subpath confirms the path.\nFinding a file borg list /path/to/repo::archive-name | grep savegame borg list /path/to/repo::archive-name --pattern \u0026#39;**/*filename*\u0026#39; Across every archive, slow on a large repo but thorough:\nfor a in $(borg list /path/to/repo); do echo \u0026#34;== $a ==\u0026#34; borg list /path/to/repo::$a | grep filename done Borg has no queryable index: searches read archive listings, so --pattern beats piping through grep.\nAfter a partial deletion An archive was deleted (borg delete repo::archive): only metadata goes; chunks survive if another archive references them. No undelete command, and the rest stay restorable.\nChunks were deleted — files removed from repo/data, or repository corruption. Whatever chunk is gone, the file needing it is gone. What helps:\nborg check --repair /path/to/repo borg mount /path/to/repo /mnt/borg --repair reconstructs missing index files and salvages orphaned chunks, not conjure deleted ones; mounting extracts what is accessible. Carving raw chunk files needs external tooling, incomplete by nature.\nRecovery is bounded by surviving chunks: an offsite copy is the only real answer, and prune deserves delete\u0026rsquo;s care.\nErrno 5 during a manifest write File \u0026#34;.../borg/repository.py\u0026#34;, line 1467, in get_write_fd self.close_segment() ... OSError: [Errno 5] Input/output error The traceback ends in fdatasync on a segment file during the manifest write — storage failing, not Borg: failing disk, full filesystem, vanished network mount, unreliable destination. Check its filesystem and write path (/mnt/server/... hints the mount); borg check shows whether the repository survived the truncation.\nSilent early exits Borg killed leaves no error of its own; exit status and system log are the evidence:\nOOM killer. dmesg | grep -i kill, and watch memory during the run — many small files are memory-hungry. Timeout. A systemd service with TimeoutSec=, or a cron wrapper, stops it: systemctl cat borg-backup.service. Disk full or quota. df -h, quota, borg create --progress to see where it stops. Bad exclude files. An empty or malformed --exclude-from / --patterns-from leaves nothing to back up, so it finishes quickly and cleanly. borg create -v --list ... Verbose output separates \u0026ldquo;ran and found nothing\u0026rdquo; from \u0026ldquo;was stopped part way\u0026rdquo; better than the exit code.\n","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/borg-restore-and-recovery/","title":"Borg restore, partial loss, and silent exits"},{"content":"What takes the space borg list --format prints raw byte sizes by field: sort and tail to find heavy files:\nborg list --format \u0026#39;{size:\u0026gt;12} {path}{NL}\u0026#39; ~/backup::desktop-2026-05-06 | sort -n | tail -50 1200000000 /home/user/videos/movie.mkv 420000000 /home/user/photos/raw.zip numfmt --to=iec reads them as 1.2G, 850M, 420M; such findings usually mean a directory to exclude, not a Borg problem.\nVerifying without restoring borg check /path/to/repo borg check user@host:/path/to/repo borg check --verify-data /path/to/repo::archive-name borg check --repository-only /path/to/repo borg check verifies repository structure, metadata and archive consistency. --verify-data also reads and hashes contents — much slower, the only proof the data is intact. --repository-only limits the check to the repository, an archive argument to that archive.\nA cheap smoke test without full verification:\nborg list /path/to/repo::archive-name borg extract --dry-run /path/to/repo::archive-name some/path/in/archive What prune keeps borg prune -v --list --progress --keep-weekly=10 --keep-monthly=4 --keep-yearly=2 prune deletes whole archives per the retention rules, keeping the most recent of each period:\nFlag Keeps --keep-weekly=10 newest of the last 10 calendar weeks --keep-monthly=4 newest of the last 4 months --keep-yearly=2 newest of the last 2 years Twenty January backups collapse to one kept archive for the month unless a weekly or daily rule retains more. Deletion is dedup-safe: blocks a kept archive references stay, the rest freed.\nWithout --keep-daily, daily granularity is not retained; retention is the union of named rules, not a superset.\nArchive names are free-form No naming pattern is enforced: a new one for future backups leaves old archives untouched and coexisting:\nborg create /path/to/repo::myhost-{now:%Y-%m-%d-%H%M} /home/user Two caveats: the new pattern must not collide with existing names, and anything filtering by prefix or glob — prune, list, check in a script, borgmatic — must be told. Renaming old archives is optional:\nborg rename /path/to/repo::oldname newname \u0026ndash;filter is output, not content --filter=AME selects which files Borg prints while running, not which are included:\nLetter Meaning A added — new in this archive M modified since the last archive E existing — unchanged borg create --filter=AME ... A /home/user/newfile.txt M /home/user/config.cfg E /home/user/picture.jpg The snapshot is identical whatever the filter: AME watches progress as a change log, and -v/--list add detail.\n","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/borg-retention-and-inspection/","title":"Borg: what a backup contains, and what retention keeps"},{"content":"Channeling is water taking the path of least resistance through the puck: it finds a crack or loose patch and drives a channel through it instead of pushing evenly through the coffee. Parts of the puck get too much water and turn bitter, parts too little and stay sour. Both arrive at once — the signature.\nA bottomless portafilter does not cause it. It removes the spouts that were hiding it.\nSigns During the shot:\nflow starts on one side, or jumps between spouts jets or sprays off to the side instead of a single stream a good cone of syrup suddenly turning pale and thin (\u0026ldquo;blonding\u0026rdquo; early) the shot accelerates as the puck opens up In the puck afterwards: holes, cracks, or an uneven surface. A soupy puck is consistent with channeling but proves nothing on its own.\nIn the cup: bitterness and sourness together, thin body, no crema, a different result every pull.\nA correct extraction is a centred, syrupy cone — \u0026ldquo;rat tail\u0026rdquo; — with no spray.\nCauses Bad distribution. Voids, mounds and uneven density — what WDT fixes. Inconsistent tamp — tilted, or pressure that differs per pull. Grind too coarse. A less resistant puck opens channels more easily. Too fine and a bad tamp also cracks it. Wrong dose for the basket. Under-dosing leaves headspace and the puck expands unevenly. Stale coffee. No CO₂ left, so the puck behaves unpredictably. High pressure. At 9 bar the water forces its way through any weak point. Dropping to ~6–7 bar is the biggest improvement on a stock Gaggia Classic, and helps most with light and medium roasts. Dirty or worn basket and shower screen. Clogged holes redistribute the flow. Uneven grind. A grinder producing boulders and fines gives the water a pre-made path. WDT cannot fix particle size. Why medium roast still channels with WDT WDT solves distribution and nothing else. Medium roast sits in the awkward middle: not as tolerant as dark, not as forgiving as it looks, so the remaining variables — particle uniformity, tamp consistency, dose versus headspace, pressure, absence of pre-infusion — still open channels. The fixes that matter most:\ngrind finer, just before the shot chokes reduce pressure toward 6–7 bar make the tamp identical every time match dose to basket capacity Small upgrades that help: puck screen, dosing funnel, levelling before tamping.\nWhy dark roast hides it Channeling happens with dark roast too; it is less visible and less punishing:\nthe brittle beans shatter into fines that plug micro-channels they are more soluble, so the rest of the puck extracts acceptably anyway the puck forms more uniformly, masking distribution errors the strong roast flavour covers the defect So a dark roast can drink fine while the technique is wrong — and then a medium roast from the same roaster tastes broken. Forgiving makes it a poor training ground.\nReferences Barista Hustle: channeling ","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/channeling/","title":"Channeling: what the naked portafilter is telling you"},{"content":"Why forwarded mail lands in spam Cloudflare Email Routing forwards mail from you@yourdomain.com to an inbox. As a forwarder it becomes the sender Gmail sees, so the original domain\u0026rsquo;s SPF and DKIM results stop matching the delivering server and Gmail defaults to suspicion. Records must be relaxed, not strict.\nv=spf1 include:_spf.mx.cloudflare.net ~all v=DMARC1; p=none; sp=none; aspf=r; adkim=r; rua=mailto:dmarc-reports@yourdomain.com aspf=r and adkim=r stop a forwarded message being penalised for a check it cannot pass. p=none is the starting policy; quarantine or reject comes later, once delivery is stable.\nCloudflare does not DKIM-sign forwarded mail — it did not author it — so the sender\u0026rsquo;s signature usually fails after forwarding: expected, not a misconfiguration.\nFailures are silent A failed forwarded message produces no non-delivery report; NDRs are not sent back to the original sender. The dashboard counts dropped, forwarded and successful messages — retrospective, not alerting.\nOptions, by effort:\ncheck the dashboard\u0026rsquo;s routing insights periodically, learning about failures late route inbound mail to a Worker, not a destination address; the Worker attempts delivery and reports failure — a webhook to Slack or a second inbox use a real mail provider where delivery must be guaranteed — forwarding-only services have no bounce handling by design The third is the honest answer for mail that matters.\n","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/cloudflare-email-routing/","title":"Cloudflare Email Routing into Gmail, and the alerts it does not send"},{"content":"\u0026ldquo;Dialing in\u0026rdquo; comes from the analog dial — the knobs on an old radio, later an espresso machine\u0026rsquo;s pressure gauge. It means tuning the variables until the shot tastes right.\nThe variables Grind size is the most sensitive and the one you move; the rest are constants you hold still:\nVariable Value Dose (double basket) 17–18 g Yield 36–40 g Time 25–30 s Ratio 1:2 A Gaggia Classic Pro with its stock non-pressurized 58 mm basket works at 18 g in, 36 g out. Brew temperature is the fifth variable, and without a PID it is not under your control.\nThe order matters: change grind, pull, taste, change grind again. Adjusting two things at once tells you nothing about which did it.\nWeigh, do not scoop Volume and weight diverge because the bean is not a solid: it is full of micro-cells, and how much space a given weight of grounds takes depends on:\nRoast level — the biggest factor. Dark roast spends longer in the heat, loses water, expands and goes brittle, so the same 18 g takes visibly more room in the basket than a light roast. CO₂ — fresh beans hold gas that pushes particles apart. Particle shape — fines pack down, a coarse grind stays airy. Variety and altitude — Robusta is denser than Arabica; high-grown is denser than low-grown. Consequences: a dark roast may overflow a basket a medium roast fills exactly, and the same scoop can land at 14 g one week and 18 g the next. Weigh the dose and the yield.\nSingle vs double Grounds Yield Single 7–9 g 25–30 ml Double 14–18 g 50–60 ml A double is not a longer pull of the same puck — it is twice the coffee. A single leaves so little in the basket that the water flows unevenly more easily, needing more precision for a less consistent result. Machines are built around the double: water distribution, pressure and basket size all assume it. Splitting a double into two cups is the usual answer when two people want one coffee each.\nPractical rule: pull doubles, and treat the single basket as a curiosity.\nReferences Gaggia Classic Pro manual Barista Hustle: espresso compass ","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/espresso-dialing-in/","title":"Dialing in espresso: dose, yield, time"},{"content":"fprintd owns the reader and answers authentication requests; the attempt count before password fallback is a parameter on the PAM module line that calls it.\nIs the reader or the PAM stack the problem fprintd-list \u0026#34;$USER\u0026#34; fprintd-verify systemctl status fprintd lsusb fprintd-verify success means the reader and stored prints are fine; the failure is the desktop\u0026rsquo;s PAM configuration. Fedora generates PAM files from profiles:\nauthselect current authselect check grep -R \u0026#34;pam_fprintd\u0026#34; /etc/pam.d/ /etc/authselect/ 2\u0026gt;/dev/null That grep lists every service accepting a fingerprint, and by omission those that do not.\nRaising the attempt limit ⚠️ Back up first. A malformed auth line locks you out of the session you edit from; recovery needs a TTY or rescue boot.\nsudo cp /etc/pam.d/system-local-login /etc/pam.d/system-local-login.bak sudo cp /etc/pam.d/sudo /etc/pam.d/sudo.bak Change the module line — both max_tries= and max-tries= appear in documentation and the module accepts that name:\nauth sufficient pam_fprintd.so max_tries=3 timeout=10 sufficient ends authentication on success; failure falls through to the next method. The file depends on the failing prompt:\nFile Affects /etc/pam.d/system-local-login local login on Arch /etc/pam.d/system-auth every service that includes it, on Arch /etc/pam.d/gdm-password GDM /etc/pam.d/sudo sudo /usr/lib/pam.d/polkit-1 desktop authentication dialogs /usr/lib/pam.d/polkit-1 is polkit\u0026rsquo;s stack: the prompt when an unprivileged process asks something privileged — mounting a drive, changing network settings. It includes system-auth by default, so a max_tries there covers all polkit prompts; to limit it to polkit, bypass the include:\nauth sufficient pam_fprintd.so max_tries=3 timeout=10 auth include system-auth Recovery Failure mode: a session that will not unlock because the PAM stack accepts nothing.\nPress your password first — fingerprint is usually additional, not a replacement. Esc sometimes switches the GDM auth UI. Ctrl+Alt+F3 (or F2, F4, F5) reaches a text login on a TTY; the graphical session keeps running. loginctl unlock-session loginctl list-sessions sudo cp /etc/pam.d/system-local-login.bak /etc/pam.d/system-local-login On a TTY, Ctrl+C aborts the fingerprint prompt and falls back to the password; graphical greeters do not.\nKeep max_tries low: each attempt is another chance for a replicated or forged print — the Arch Wiki\u0026rsquo;s warning about fingerprint for su and sudo. Fingerprint as sufficient for sudo without a TTY check is a privilege-escalation path; the documented guard is a pam_succeed_if rule matching service in sudo:su:su-l tty in :unknown before the fingerprint line — the pattern in Arch setup notes.\nReferences Arch Wiki: fprint fprintd pam_fprintd(8) ","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/fingerprint-pam/","title":"Fingerprint login: PAM decides who gets asked"},{"content":"Editing /etc/resolv.conf in a running container does not stick — Docker overwrites it on start and on networking changes. Configure the resolver at one of three levels:\nPer container:\ndocker run --dns=1.1.1.1 --dns=8.8.8.8 myimage docker exec -it \u0026lt;container\u0026gt; cat /etc/resolv.conf Per Compose service:\nservices: app: image: myimage dns: - 1.1.1.1 - 8.8.4.4 Daemon-wide, in /etc/docker/daemon.json, then sudo systemctl restart docker:\n{ \u0026#34;dns\u0026#34;: [\u0026#34;1.1.1.1\u0026#34;, \u0026#34;8.8.8.8\u0026#34;] } --dns on the container overrides the daemon setting.\nBind-mounting your own file over the path is the fourth option, and worst:\ndocker run -v /path/my-resolv.conf:/etc/resolv.conf myimage It survives restarts but stops Docker managing the file, breaking anything depending on Docker\u0026rsquo;s own name resolution inside the container.\nUsual case: a container that must query a LAN resolver — AdGuard Home or Pi-hole — where the daemon-wide setting is wrong, since it sends every container there. See DNS troubleshooting for a container timing out against an internal upstream.\n","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/docker-dns/","title":"Giving containers their own DNS"},{"content":"Stow symlinks a central config directory into place; nothing is copied, so editing ~/.zshrc edits the repository file.\nsudo apt install stow # Debian, Ubuntu sudo pacman -S stow # Arch brew install stow # macOS Each package mirrors the target:\n~/dotfiles/ ├── zsh/.zshrc ├── git/.gitconfig └── nvim/.config/nvim/init.lua cd ~/dotfiles stow zsh stow zsh git nvim stow */ Exact structure: zsh/zshrc links nothing, zsh/.zshrc does. Preview with a dry run:\nstow -n -v zsh What it links in .config nvim/.config/nvim/init.lua in the repository makes stow nvim produce:\n~/.config/nvim -\u0026gt; ~/dotfiles/nvim/.config/nvim Stow links nvim inside .config, not .config itself, avoiding an existing directory; with ~/.config absent it may link all of it.\nTo force targeting .config, restructure and say so:\ndotfiles/nvim/nvim/init.lua stow --target=$HOME/.config nvim -t targets any other destination, e.g. bin/ to ~/scripts:\nstow -t ~/scripts bin Conflicts and adoption An existing target makes Stow refuse rather than overwrite: move it aside, or let Stow move it into the repository:\nmv ~/.zshrc ~/.zshrc.bak stow --adopt zsh --adopt is convenient once, dangerous run blindly: it overwrites the repository copy with the home directory version.\nUndoing stow -D zsh # remove the links stow -D . stow -R zsh # restow: delete and recreate Deleting the symlink by hand is equivalent; the repository file is untouched, and Stow only creates symlinks. Check first: rm deletes files as readily as links:\nls -l ~/.config # the -\u0026gt; arrow marks a symlink stow -D is safer when many links exist and you do not know which Stow created.\nMultiple machines Layout: a shared base plus per-machine packages:\ndotfiles/ ├── common/ ├── machines/{desktop,laptop,server}/ └── install.sh HOST=$(hostname) stow common [ -d \u0026#34;machines/$HOST\u0026#34; ] \u0026amp;\u0026amp; stow \u0026#34;machines/$HOST\u0026#34; common always applies, the machine package only where it exists; OS variation goes in separate packages, not conditionals in one file. .stow-local-ignore excludes files from a package when some should not be linked.\n","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/gnu-stow-dotfiles/","title":"GNU Stow for dotfiles"},{"content":"Green coffee is processed three ways: washed (fruit and mucilage stripped before drying), natural (dried whole inside the fruit), or honey — depulped, with some mucilage left on the bean through drying. Honey sits between washed and natural, and the colour name says how much sticky layer was kept.\nType Mucilage kept Cup White Honey Little Cleaner, lighter body Yellow Honey Some Sweet, balanced acidity, medium body Red Honey More Sweet, syrupy mouthfeel Black Honey ~100% Heavy, creamy, molasses and toffee Why Black Honey is expensive The mucilage is sugar, so the bean dries inside a sticky food. The producer must:\nselect ripe fruit by hand at harvest depulp without washing the mucilage off dry slowly — up to 15 days or more turn them regularly so they do not mould keep them out of direct sun, often under black canvas; control humidity closely That is labour and risk, paid for in the price. The process also uses far less water than a fully washed coffee, and gives the producer a knob: more mucilage means more sweetness and body, less means more clarity — tuning the cup before roasting.\nWhat it tastes like Sweetness with weight behind it — molasses, toffee, caramel, dried fruit — softer, better-balanced acidity than a washed coffee from the same farm, fruit in the background, and a rounded, syrupy body.\nReferences Specialty Coffee Association: processing ","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/black-honey-process/","title":"Honey and Black Honey: how much mucilage stays on the bean"},{"content":"The one difference that matters Command Copies new/changed Deletes on target rclone copy source/ remote:target/ yes no rclone sync source/ remote:target/ yes yes sync mirrors the destination exactly: delete a save locally, run sync, and the remote copy goes too. Wrong default for backups — copy leaves anything the source no longer has.\nAlways preview a destructive run:\nrclone sync /path/to/source remote:path/to/dest --dry-run -v --delete-before changes when deletions happen relative to the transfer, not whether they happen. copy with --delete-excluded is the middle option when exclusions should also propagate.\nEmptying a target rclone purge --dry-run gdrivecrypt:backup/containers rclone purge gdrivecrypt:backup/containers rclone delete gdrivecrypt:backup/containers purge removes everything recursively including the directory; delete empties it and keeps the directory. Mirroring an empty directory with sync achieves the same as purge and does more work; --dry-run is the cheap way to be sure which you want.\nEncrypting a local folder Encryption is a remote that wraps another remote, including a local one. Two entries are needed: a plain local remote pointing at the directory that will hold ciphertext, and a crypt remote on top of it.\nrclone config new remote → name: localcryptraw → storage: local → path: /path/to/encrypted new remote → name: localcrypt → storage: crypt → remote: localcryptraw:/ The crypt remote asks for a password and a salt password. Commands through localcrypt: then encrypt on write and decrypt on read, so plaintext never exists on disk outside the source. Losing that password loses the data — no recovery path.\nHow tools use it Ludusavi treats rclone as a backend and builds the command from your configuration, not a fixed one: rclone copy by default, rclone sync when mirroring is enabled, with --progress, --stats, --transfers and whatever extra arguments you configured.\nThe exact invocation is in its log — read it when a sync deletes something unexpected:\n~/.config/ludusavi/logs/ # Settings → Cloud → enable logging, then run a backup ","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/rclone-sync-and-encryption/","title":"rclone: sync versus copy, deletions, and local encryption"},{"content":"/etc is 10–50 MB of mostly text: take the whole directory, not cherry-picked configs:\npacman -Qqe \u0026gt; ~/pkglist.txt # explicitly installed pacman -Qqm \u0026gt; ~/aur-pkglist.txt # from the AUR Those two lists make a rebuild an hour, not a weekend. Easy to forget beyond /etc: ~/.config, the shell dotfiles, ~/.ssh, ~/.gnupg, ~/.gitconfig, the bootloader config — /etc/default/grub for GRUB, /boot/loader/ for systemd-boot, outside /etc.\nOne Borg run covering system, user config and package state:\nborg create \\ --stats \\ --compression zstd,6 \\ --exclude \u0026#39;/home/*/.cache\u0026#39; \\ --exclude \u0026#39;/home/*/.local/share/Trash\u0026#39; \\ ::arch-{now:%Y-%m-%d} \\ /etc \\ /home/$USER/.config \\ /home/$USER/.local/share \\ /home/$USER/.bashrc \\ /home/$USER/.zshrc \\ /home/$USER/.profile \\ /home/$USER/.ssh \\ /home/$USER/.gnupg \\ /home/$USER/.gitconfig \\ /home/$USER/pkglist.txt \\ /home/$USER/aur-pkglist.txt Preview and retention:\nborg create --dry-run ::test /etc borg prune --keep-daily=7 --keep-weekly=4 --keep-monthly=6 --stats ⚠️ Restoring /etc wholesale onto a fresh install is the mistake: some of it is machine-specific or generated at install time. Diff first, then apply what you want:\nrsync -a --dry-run etc-backup/ /etc/ diff -ruN etc-backup/ /etc/ pacman -Qkk verifies tracked files against their package — the check for an altered restore.\nWhich bootloader is in use archinstall picks by firmware; the bootloader matters when restoring /boot:\nScenario Bootloader UEFI (the default path) systemd-boot BIOS / legacy GRUB Manual selection either, or none systemd-boot installs to /boot/EFI/systemd/, entries under /boot/loader/entries/arch.conf. Confirm, do not assume:\nbootctl status bootctl reporting systemd-boot information means systemd-boot; if not found, GRUB or a manual setup.\nReferences BorgBackup bootctl(1) ","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/arch-system-backup/","title":"What to back up on Arch, and a Borg command that does it"},{"content":"zsync delivers a new version of a file you already have. The server publishes a small .zsync sidecar of block checksums for the target file:\nubuntu.iso ubuntu.iso.zsync The client reads it, examines the local file, and fetches only changed blocks:\nzsync http://example.com/ubuntu.iso.zsync For a 4 GB ISO with 300 MB of changes, that is roughly 300 MB transferred instead of 4 GB.\nAt scale the key property: the server needs nothing special — an ordinary web server or CDN, and no copy of your old file:\nzsync rsync Old file on the client yes yes Old file on the server no usually Works over HTTP yes not normally Server-side software required no usually Best for public downloads general synchronisation Closest to a torrent client reusing pieces it already has, but from a local file and over plain HTTP rather than peer-to-peer.\nAppImages An .appimage.zsync is the same sidecar for an AppImage, consumed by AppImageUpdate:\nAppImageUpdate YourApp.AppImage It downloads the .zsync, compares it with the local AppImage, fetches changed chunks and rebuilds the file. The sidecar is never used directly; deleting it breaks delta updates for that AppImage.\nReferences zsync AppImageUpdate ","date":"2026-04-17T00:00:00Z","dateFormatted":"17 Apr 2026","permalink":"https://genezini.com/garden/zsync-delta-updates/","title":"zsync: updating a big file over plain HTTP"},{"content":"Two sysctls, two limits, both per user:\nSetting Limits Default fs.inotify.max_user_instances inotify instances one user may create 128 fs.inotify.max_user_watches watches one user may attach 8,192 An instance is one inotify_init() / inotify_init1() file descriptor — an fsnotify group, an event queue, the descriptor, and indexing structures for its watches. A watch is one attachment to an inode, bookkeeping linking watch, instance and inode.\nMany tools opening listeners exhaust instances; few listeners over many files exhaust watches.\nCeilings, not reservations Raising either value costs nothing — upper bounds, not preallocated memory — but every instance and watch consumes kernel memory:\nper instance, a modest fixed cost plus an event queue growing with pending events per watch, real bookkeeping, roughly ~1 KB on modern 64-bit Linux (estimates from ~540 bytes on 32-bit upward) At 524,288 watches the worst case is 500 MB of kernel memory — not reserved, but real if used. Watch count, not instance count, is usually the memory driver.\nWhy a monorepo gets there inotify reports created, edited, deleted and renamed files; every tool reacting to a source change uses it: the Next.js dev server, Turbopack or webpack watchers, Turborepo\u0026rsquo;s task runner, Vitest in watch mode, the Biome and TypeScript language servers, the editor, and pnpm\u0026rsquo;s workspace linking.\nA pnpm + Turborepo monorepo with two Next.js apps, 23 shared packages and a 4.9 GB node_modules multiplies that: each app and package is a tree of directories, each tool opens its own instance, and pnpm\u0026rsquo;s symlinks reach the same real files through several paths. Raising max_user_instances from 128 to 512 addresses the per-tool listeners, not the file count, often fixing the symptom when max_user_watches is already generous.\nWhat failure looks like When a limit is hit the watcher cannot be installed and the tool degrades:\nENOSPC in the output — the \u0026ldquo;too many open files\u0026rdquo; family, despite plenty of disk hot reload stops firing, or fires for some packages and not others the dev server falls back to polling — works, but costs CPU language server features (completions, diagnostics) become slow or stop updating Nothing crashes; the environment just becomes unreliable.\nReducing the demand exclude node_modules from watching on both sides — the bundler\u0026rsquo;s watchOptions.ignored and the editor\u0026rsquo;s file watcher watch only the packages a task needs, not the whole workspace root poll deliberately for trees the watcher cannot handle (network mounts, some VM shared folders), accepting the CPU cost fewer listeners: one dev server instead of one per app when possible Raising the limits is the cheap first move on a workstation. On a shared server or in containers the values are a resource question: the ceiling is per user, so a high value lets any process in that user consume it.\n","date":"2026-04-14T00:00:00Z","dateFormatted":"14 Apr 2026","permalink":"https://genezini.com/garden/inotify-limits/","title":"inotify limits: instances, watches, and what a monorepo exhausts"},{"content":"develop and main are usually protected — no reset, no force-push — so undoing means new commits on top. git revert needs a commit range and a -m parent decision per merge commit: many chances to pick wrong.\ngit revert 4dbcaf5419d5b5aa19776d7ed733a163cacd95fe..dbffeb4da345fd0c66b0b7a8193a6582b409cd4a The fallback is a second clone: target branch checked out elsewhere, everything but .git deleted, the good branch\u0026rsquo;s contents copied in, commit — works, miserable.\nThe one-liner git restore copies files out of any revision into the working tree without touching the index, unlike git checkout or git reset, which move it — so the desired tree pulls in on the protected branch:\ngit fetch --all git restore --source=origin/main . A file or glob narrows it:\ngit restore --source=origin/main index.html restore only writes the working tree, so the revert still needs its commit:\ngit add -A git commit -m \u0026#34;Revert to origin/main\u0026#34; No merge parents, no commits to enumerate, and the history stays append-only — the reason for protection.\nReferences About protected branches (GitHub) git-revert git-restore ","date":"2025-06-10T00:00:00Z","dateFormatted":"10 Jun 2025","permalink":"https://genezini.com/garden/git-restore/","title":"Reverting a protected branch with git restore"},{"content":"Introduction Keeping a professional resume updated and adaptable for various applications can be a challenging. This post demonstrates how leveraging Markdown for content and Pandoc for conversion simplifies the process, enabling you to maintain a single, clean source file that effortlessly transforms into polished PDF, DOCX, or HTML outputs.\nWhy Markdown and Pandoc? When it comes to resume management, using Markdown and Pandoc offers several advantages:\nVersion Control: Your resume becomes easily trackable in Git and you can use tags or branches to preserve the versions that were sent to a job posting; Simple Syntax: Markdown is simple, human-readable syntax makes editing and maintaining your resume straightforward; Multiple Formats: Update your resume in one central Markdown file and generate multiple formats (PDF, DOCX, HTML, etc.). The source can even be rendered directly in platforms like GitHub; Separation of Concerns: Keep your resume content clean in Markdown while managing formatting externally through templates. This ensures consistent styling across all generated formats, unlike manually formatted documents where styles can drift over time; Multi-language Support: Facilitate multi-language resumes by managing different language versions of your content more easily; Automation with Filters: Extend functionality with Pandoc filters (written in Lua or Python) to automate tasks like citations, custom formatting, or conditional content inclusion. Setting Up the Environment To get started, you\u0026rsquo;ll need Pandoc and a PDF rendering engine like LaTeX, wkhtmltopdf, or WeasyPrint. Note that each engine has specific dependencies; for instance, using LaTeX necessitates installing numerous packages (e.g., amsfonts, amsmath, graphicx, hyperref, babel, etc.).\nTo avoid cluttering your system with numerous dependencies, consider running Pandoc within a Docker container. This approach isolates the environment while still allowing easy access to local templates, fonts, and assets via Docker volumes.\nThat said, in this post, I won\u0026rsquo;t cover installing Pandoc and will focus on using it in a container.\nPandoc has several container image variations (here is the official list), but I use pandoc/extra, that comes with LaTeX and dependencies installed.\nPull the image:\ndocker pull pandoc/extra And every time you run Pandoc, use a docker run --rm to start a container and remove it afterwards:\ndocker run --rm -v \u0026#34;$(pwd):/data\u0026#34; -u $(id -u):$(id -g) pandoc/extra {pandoc-params} Example:\ndocker run --rm -v \u0026#34;$(pwd):/data\u0026#34; -u $(id -u):$(id -g) pandoc/extra Resume.md -o Resume.pdf ℹ️ For simplicity\u0026rsquo;s sake, for the rest of the post I\u0026rsquo;ll use just the pandoc {params} command. Creating Your Resume in Markdown Begin by creating a resume.md file. Here’s a fundamental structure to get you started:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 # John Doe ## Software Engineer \u0026gt; [john@doe.com](mailto:john@doe.com) | \u0026gt; [LinkedIn](https://linkedin.com/in/johndoe) | \u0026gt; [GitHub](https://github.com/johndoe) ## Summary Experienced software engineer with 8+ years of experience in full-stack development... ## Experience ### Senior Software Engineer | Tech Corp *2020 - Present* - Led development of microservices architecture serving 1M+ users - Mentored junior developers and conducted code reviews - Implemented CI/CD pipelines reducing deployment time by 40% ### Software Engineer | Start Up Inc *2017 - 2020* - Developed and maintained customer-facing web applications - Improved application performance by 60% ## Skills - **Languages**: Python, JavaScript, C#, SQL - **Frameworks**: Django, React, .NET Core - **Tools**: Docker, Kubernetes, AWS Generating Your Resume Use the following Pandoc commands to convert your Markdown resume into various formats:\nTo PDF pandoc resume.md -o resume.pdf To DOCX pandoc resume.md -o resume.docx To HTML pandoc resume.md -o resume.html -s Templates Pandoc templates offer diverse styles and functionalities. Select one that matches your desired aesthetic and customization needs. Popular choices include Eisvogel for LaTeX/PDF and various options for HTML/CSS.\nIf you intend to generate in more than one format, you need to chose a template that support all of them or use one template for each format.\nUsing a built-in or installed template (e.g., Eisvogel for PDF) To use a built-in template pass its name in the template parameter:\npandoc resume.md -o resume.pdf --template eisvogel Using a custom template file within a container To use a custom template within a container, create a volume mapping a folder with the container and pass the relative path in the template parameter:\ndocker run --rm \\ --volume \u0026#34;$(pwd):/data\u0026#34; \\ --volume \u0026#34;$(pwd)/custom-templates:/templates\u0026#34; \\ --user \u0026#34;$(id -u):$(id -g)\u0026#34; \\ pandoc/extra \\ resume.md \\ -o resume.pdf \\ --template /templates/my-custom-template.latex Frontmatter parameters Pandoc utilizes variables defined in a YAML metadata block (frontmatter) at the beginning of your Markdown file. Templates often define custom variables for controlling layout, styling, or adding header/footer information – consult the specific template\u0026rsquo;s documentation for details.\nVariables for the PDF LaTeX engine Variables for the Eisvogel template. Here is an example of variables used by Pandoc:\n1 2 3 4 5 6 7 8 9 --- title: \u0026#34;The Document Title\u0026#34; author: [Example Author, Another Author] date: \u0026#34;2017-02-20\u0026#34; keywords: [Markdown, Example] --- ... Here is the actual document text... Custom Styling (with LaTeX) For fine-grained control PDF output via LaTeX, you can include custom LaTeX code snippets. Create a file (e.g., styling.tex) with your customizations:\n1 2 3 4 5 6 7 8 9 10 11 12 % Adjust line spacing \\usepackage{setspace} \\setstretch{1.1} % Customize hyperlink color \\usepackage{xcolor} \\usepackage{hyperref} \\hypersetup{ colorlinks=true, linkcolor=blue, urlcolor=blue } Then, include this file during conversion using the --include-in-header flag:\npandoc resume.md -o resume.pdf --template eisvogel --include-in-header=styling.tex Similarly, for HTML output, you can provide a custom CSS file using the --css flag.\nTips and Best Practices Keep it Simple: Markdown is meant to be readable. Focus on content structure; Version Control: Use Git to track changes, experiment with branches, and tag versions sent to employers; Separate Content and Style: Manage styling externally using LaTeX includes/templates for PDF or CSS for HTML to keep your Markdown focused on content; Verify Output: Always check how your resume looks in each format (PDF, DOCX, HTML) to ensure correctness and the desired appearance; Use Templates and Variables: Leverage different templates and use metadata variables and Pandoc filters to easily generate tailored resume versions for specific job applications without duplicating the core content. Exploring More Templates Beyond the popular Eisvogel template, many other options exist on GitHub, offering different styles and features. Some uses YAML instead of Markdown for content.\nHere are a few examples:\njohn-bokma/resume-pandoc: LaTeX-based, uses YAML metadata; roguh/pandoc-resume: Simple template with scripts for PDF, DOCX, and HTML generation. Includes CSS and reference DOCX; plain-plain-text/simple-cv: Minimalist template using Bootstrap for HTML and memoir for LaTeX/PDF; florianschwanz/awesome-cv-pandoc: Integrates the well-regarded \u0026ldquo;Awesome CV\u0026rdquo; LaTeX template with Pandoc; bmschmidt/CV-pandoc-healy: Focuses on separating YAML data from LaTeX styling; kjhealy/pandoc-templates: A collection of Pandoc templates for generating HTML, LaTeX, and PDF documents samijuvonen/pandoc_resume_template: Uses YAML for content, supports multiple outputs (PDF, HTML, Markdown, TXT) via XeLaTeX and includes a Makefile. When choosing a template, consider its output formats, customization options, dependencies, and documentation.\nReferences Pandoc Documentation Markdown Guide LaTeX Templates GitHub Topic: pandoc-templates - Discover more templates. ","date":"2025-04-28T08:00:00-03:00","dateFormatted":"28 Apr 2025","image":"https://genezini.com/blog/p/creating-a-maintainable-resume-with-markdown-and-pandoc/cover.webp","permalink":"https://genezini.com/blog/p/creating-a-maintainable-resume-with-markdown-and-pandoc/","title":"Creating a Maintainable Resume with Markdown and Pandoc"},{"content":"Introduction Setting up a home lab is an incredible journey that opens up countless learning opportunities in system administration, networking, security, and modern deployment practices. While it might seem daunting at first, the knowledge gained is invaluable for any tech enthusiast or IT professional.\nIn this post, I\u0026rsquo;ll share a practical overview of how I use my home lab, highlighting the real-world projects I\u0026rsquo;ve tackled and the key lessons learned along the way. From self-hosting essential services to experimenting with automation, networking, and security, I\u0026rsquo;ll focus on the hands-on experiences that have shaped my understanding of home server management and modern IT practices.\nWhy Set Up a Home Lab? Before diving into the technical aspects, let\u0026rsquo;s explore why you might want to invest time in setting up a home lab:\nHands-on Learning and Career Development: Theory is great, but nothing beats practical experience. A home lab provides a safe environment to experiment and learn from mistakes. The skills you develop are directly applicable to enterprise environments, making you more valuable as an IT professional. I\u0026rsquo;ll dive into this learning topic below;\nControl Over Your Data: Self-hosting gives you complete control over your data and privacy, reducing dependency on third-party services. Some examples are:\nRunning VaultWarden (a self-hosted Bitwarden alternative) locally, which keeps your password data under your control and reduces exposure to online threats. By self-hosting, you can avoid risks such as polymorphic extension attacks that target cloud-based password managers; Running Adguard Home to provide DNS over HTTPS (DoH), which encrypts your DNS queries and prevents your internet provider from seeing or logging your browsing activity. This enhances privacy and security for all devices on your network. Cost Efficiency: Although setting up a home lab requires some upfront investment in hardware, it can be significantly more cost-effective over time compared to subscribing to multiple cloud services. By self-hosting solutions like Nextcloud for file storage and synchronization, or Immich for photo management, you gain virtually unlimited storage, constrained only by your own hardware, without recurring subscription fees. This approach allows you to scale your storage as needed, paying only for additional drives or upgrades, and often results in substantial long-term savings;\nCustomization: A home lab lets you tailor every aspect of your environment to your unique needs—something rarely possible with commercial solutions. For example, Home Assistant is a highly customizable alternative to Google Home or Apple HomeKit. It supports thousands of integrations, allowing you to connect and automate devices exactly as you want. You can create complex automations, dashboards, and routines that fit your lifestyle, rather than being limited by the features or restrictions of proprietary platforms;\nGives You New Ideas and Insights: Experimenting in your own environment exposes you to new technologies, workflows, and problem-solving approaches that you might not encounter otherwise. This often leads to creative solutions and a deeper understanding of IT concepts;\nIt\u0026rsquo;s Fun: Building and managing your own home lab is genuinely enjoyable. It’s a rewarding hobby that lets you tinker, experiment, and see immediate results from your efforts.\nWhat do you need / How to start? You can begin your home lab journey by experimenting with a virtual machine on your computer or in the cloud, which is a low-risk way to get started and learn the basics. However, the real benefits come from setting up a dedicated local server. Repurposing an old PC or laptop is a great, cost-effective option—many home lab enthusiasts start this way. For example, I began with a Chromebox: I replaced ChromeOS with Alpine Linux, and it has been running reliably for over six years. As my needs grew, I transitioned to using my old notebook as the primary server. This approach lets you make the most of existing hardware while building hands-on experience with real-world setups.\nSome home lab enthusiasts experiment with single-board computers like the Raspberry Pi, Orange Pi, or Banana Pi. Some even build clusters of multiple boards to explore distributed computing or container orchestration at a small scale. While I personally find them less cost-effective compared to repurposing old PCs or laptops—especially given recent price increases and performance limitations—they can be a fun and educational platform if you enjoy tinkering with hardware.\nWhile specialized operating systems like CasaOS or Umbrel OS designed specifically to home servers, I recommend using a standard Linux distribution. This approach offers learning opportunities and greater applicability in professional environments.\nWhat can you learn? Here are some of the key areas you can explore and learn:\nOperating Systems: Deepen your understanding of Linux distributions, command-line tools, and system internals; Containers and Orchestration: Get hands-on with Docker, Docker Compose, and Kubernetes to deploy and manage containerized applications; Service Hosting: Self-host applications like wikis, media servers, password managers, and more; Storage and File Systems: Explore RAID, LVM, and network file sharing protocols; Backup and Recovery: Develop backup/recovery strategies; Security: Practice hardening your systems, managing firewalls, setting up VPNs, and implementing authentication mechanisms; Networking: Learn about IP addressing, routing, VLANs, DNS, DHCP, and network security; Monitoring and Logging: Set up monitoring stacks (Prometheus, Grafana), alerts and log aggregation tools to keep your environment healthy; Automation: Use tools like Bash scripts and CI/CD pipelines to automate repetitive tasks and deployments. Each of these areas offers practical, hands-on experience that builds both foundational and advanced skills, making your home lab an invaluable resource for continuous learning.\nLinux Of course that if we are building a server we are going to use Linux, and this is a rabbit hole by itself: the file system structure, commands, files permissions, systemd and daemons are just some examples.\nSetting up a home lab on Linux teaches you:\nSystem administration fundamentals; Package management; Service management with systemd; Log analysis and system monitoring; Shell scripting for automation. Containers Containers are the go-to way to spin up applications. It\u0026rsquo;s easier, more secure and more maintainable. We can use compose files with Docker Compose and have the benefits of Infrastructure as Code (IaC).\nThe container journey teaches you:\nContainer lifecycle management (starting, healthy, unhealthy); Volume management and data persistence; Docker Compose for multi-container applications; Container security best practices, such as running containers as non-root users and minimizing container privileges; Image building and optimization; Docker networking concepts (host, bridge, shared networks, etc). A useful networking strategy in Docker Compose is to create a shared network for all applications, while also assigning each application its own private network for internal communication between its components. For example, a web application and its database can communicate over a dedicated private network, isolating sensitive traffic from other services. At the same time, the web application can be attached to a shared network, allowing it to interact with reverse proxies or other shared infrastructure. This approach improves security and organization by limiting cross-application access and making network policies easier to manage.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 networks: network_shared: external: true paperless: driver: bridge webserver: image: ghcr.io/paperless-ngx/paperless-ngx:latest depends_on: - db networks: - network_shared - paperless db: image: docker.io/library/postgres:15 networks: - paperless Container orchestration There are lightweight Kubernetes distributions like K3s and K3d (K3s in Docker) that are ideal for a home server, because they require fewer resources and are easier to set up compared to a full Kubernetes installation.\nWith container orchestration, you\u0026rsquo;ll learn:\nAutomated deployment and scaling of applications; Load balancing; Rolling updates and rollbacks; Managing persistent storage for containers; Use of Helm charts. This will give you some hands-on experience with kubernetes and modern DevOps practices.\nFile Sharing and Storage Understanding storage solutions is crucial for a home lab. You\u0026rsquo;ll learn about:\nDifferent file sharing protocols (SMB/CIFS and NFS) and how to expose and mount those protocols; How to mount file systems at startup and configurations to abort on or ignore errors; Storage pooling with LVM (combining multiple physical drives into a single logical volume); Software RAID for redundancy; File system permissions (and how they differ in SMB and NFS). Backup Strategies A solid backup strategy is essential to protect your data from accidental loss, hardware failure, or security incidents. In a home lab, you\u0026rsquo;ll likely experiment with several backup tools and approaches:\nBorg: Efficient, deduplicating, and encrypted backups. Great for incremental backups and remote storage. Rsync: Flexible file synchronization and backup tool, ideal for local and remote backups with minimal bandwidth usage. Rclone: Syncs files and directories to and from many cloud storage providers, making offsite backups easy. It allows layering virtual encrypted storage on top of any storage provider, ensuring data privacy and security. Key concepts you\u0026rsquo;ll learn:\nBackup scheduling and automation; Incremental and differential backups; Encryption and data integrity verification; Retention policies to manage storage usage. Experimenting with these tools will help you design a backup solution tailored to your needs and ensure your data is always safe.\nIn my setup, I use Borg to create deduplicated backups of both my servers and personal computer. Once the backups are complete, I leverage Rsync to securely sync these archives to Google Drive (encrypting them at rest) for offsite redundancy. For local photo backups, I rely on Rsync to efficiently mirror and synchronize photo directories between multiple hard drives, ensuring quick recovery and data integrity.\nReverse Proxy and SSL A crucial component for secure service access:\nNGINX or Traefik configuration; SSL/TLS certificate management; Let\u0026rsquo;s Encrypt automation. In my setup, I use NGINX Proxy Manager together with DuckDNS (pointing to my local IP, meaning it is only accessible by my local network or VPN) and Let\u0026rsquo;s Encrypt for automated SSL certificate provisioning. This combination allows me to expose my self-hosted services over HTTPS, routing all access exclusively through NGINX Proxy Manager, which acts as a reverse proxy—ensuring that only authenticated and encrypted traffic reaches my internal services.\nAuthentication and Security Security is critical in any server environment, so I\u0026rsquo;ll learn:\nOpenID and OAuth2; Single Sign-On (SSO) implementation; Network segmentation; Firewall configuration. I use Authentik to handle OpenID authentication, integrating it with NGINX to secure proxied services.\nNetworking Essential networking concepts you\u0026rsquo;ll learn:\nDNS management and local DNS resolution; Port forwarding (and why not to do it); Network troubleshooting. In my setup, I use DuckDNS to obtain a free domain name, resolving to a local IP, so it is only accessible in my local network. For local DNS resolution and caching, AdGuard Home acts as my DNS server, providing fast lookups and enhanced privacy for all devices on my network. To securely access my home lab from anywhere without exposing ports or relying on traditional port forwarding, I leverage ZeroTier—a peer-to-peer VPN solution that uses UDP hole punching to create encrypted, direct connections between devices, even behind NAT or firewalls. This combination ensures secure, reliable, and private connectivity to my services, both locally and remotely.\nMonitoring and Maintenance Keeping your services healthy:\nSystem monitoring and resource usage with Prometheus/Grafana; Application health-check monitoring; Log aggregation and analysis; Alerts configuration. For monitoring the health and performance of my home lab, I use a combination of tools:\nUptime Kuma provides real-time health checks for all my services, alerting me immediately via Telegram if any become unreachable. Prometheus collects detailed metrics from both containers and the host system, using cAdvisor for container-level insights and Node Exporter for system metrics. These metrics are visualized in Grafana dashboards, giving me a comprehensive view of resource usage and trends. I’ve set up alerting rules in Grafana to notify me via Telegram whenever critical thresholds are breached, ensuring I can respond quickly to issues. For log monitoring, I rely on Dozzle to view container logs in real time. It’s lightweight and simple, which suits my needs since I don’t require long-term log retention. Automation and Scheduling Automation is the secret sauce that transforms a home lab from a collection of services into a streamlined, low-maintenance environment. By automating routine tasks, you reduce manual effort, minimize human error, and ensure your services remain reliable and up-to-date.\nSome practical automation strategies include:\nAutomated Backups: Schedule regular backups using tools like cron, or backup software with built-in schedulers. This ensures your data is always protected without manual intervention; Scripted Automations: Use shell scripts, or CI/CD pipelines (e.g., GitHub Actions) to deploy and update your applications consistently; Scheduled Maintenance: Automate cache cleanups, and other housekeeping tasks to keep your environment running smoothly. Conclusion Setting up a home lab is an ongoing journey of learning and discovery. While it requires time and dedication, the knowledge and skills gained are invaluable. Start small, focus on one area at a time, and gradually expand your infrastructure as you learn.\nRemember that mistakes are part of the learning process - having a good backup strategy means you can experiment freely without fear of losing data.\nApplications/Tools Here\u0026rsquo;s a comprehensive list of tools mentioned in this article:\nAdGuard Home - Network-wide DNS privacy and ad blocking DuckDNS - Free dynamic DNS service Let\u0026rsquo;s Encrypt - Free SSL certificate authority NGINX - Web server and reverse proxy NGINX Proxy Manager - GUI for NGINX proxy management Traefik - Modern reverse proxy and load balancer ZeroTier - P2P VPN for secure remote access K3s - Lightweight Kubernetes distribution K3d - K3s in Docker for local development cAdvisor - Container resource usage metrics Dozzle - Real-time container log viewer Grafana - Metrics visualization and alerting Node Exporter - Hardware and OS metrics Prometheus - Metrics collection and storage Uptime Kuma - Uptime monitoring Borg - Deduplicating backup program Rclone - Cloud storage sync tool Rsync - Fast file copying and syncing Home Assistant - Home automation platform Immich - Self-hosted photo backup solution Nextcloud - File hosting and collaboration Vaultwarden - Password management server Authentik - Open-source Identity Provider focused on flexibility and versatility. References Docker Documentation r/selfhosted awesome-selfhosted selfh.st/apps Polymorphic Chrome Extensions Attack ","date":"2025-04-22T06:00:00-03:00","dateFormatted":"22 Apr 2025","image":"https://genezini.com/blog/p/down-the-rabbit-hole-of-creating-a-home-lab/cover.webp","permalink":"https://genezini.com/blog/p/down-the-rabbit-hole-of-creating-a-home-lab/","title":"Down the Rabbit Hole of creating a Home Lab"},{"content":"Introduction Single-page applications (SPA) provide better user experience by removing full-page loads, giving smooth transitions between pages and performance improvements depending on the scale of the application, but they come at some costs which may not be worth for some application types. The main cost is the added complexity of a SPA framework like Angular or Vue.\nIn this post, I\u0026rsquo;ll show how to use ASP.NET and HTMX to build single-page applications without using JavaScript and SPA frameworks.\nWhat is HTMX HTMX is a library that allows us to access modern browser features by extending HTML, without needing to use Javascript directly. It gives us access to AJAX, CSS Transitions, WebSockets and more, including extensions.\nIt simplifies making responsive apps that give our users a better experience without the complexity of Javascript.\nHTMX is really powerfull, but in this post I\u0026rsquo;ll focus on the hx-boost attribute.\nBase application I\u0026rsquo;ll use an old Razor Pages Sample Application by Damian Edwards to show how easy it is to make an SPA with ASP.NET and HTMX.\nI\u0026rsquo;ve made some changes in his sample, including upgrading from .NET Core 2.0 to .NET 7. These changes are in the branch sample-base of the sample repository: AspNetCoreSPAHtmx.\n💡 I\u0026rsquo;m using Razor pages but what I\u0026rsquo;ll show is also aplicable to ASP.NET MVC. Including the HTMX lib and menu links configurations Let\u0026rsquo;s open the _Layout.cshtml and add the reference to HTMX at the end of the \u0026lt;body\u0026gt; tag:\n1 2 3 4 ... \u0026lt;script src=\u0026#34;https://unpkg.com/htmx.org@1.9.6\u0026#34;\u0026gt;\u0026lt;/script\u0026gt; \u0026lt;/body\u0026gt; HTMX has an attribute hx-boost that enable us to transform our Multi-Page Application in a Single Page Application (SPA) with just some tweaks, making \u0026lt;a\u0026gt; and \u0026lt;form\u0026gt; tags use AJAX requests instead of page reloads.\nIt works on all the children of the element it is applied to, so let\u0026rsquo;s include it in the \u0026lt;nav\u0026gt; tag so all the menu links use AJAX:\n\u0026lt;nav class=\u0026#34;navbar navbar-inverse navbar-fixed-top\u0026#34; hx-boost=\u0026#34;true\u0026#34; hx-target=\u0026#34;#main-content\u0026#34;\u0026gt; By default, hx-boost applies the AJAX response to the \u0026lt;body\u0026gt; tag, so we need to specify the id of target (in this sample, #main-content) in the hx-target attribute.\nNow, let\u0026rsquo;s create a div with id main-content to receive the response content inside the body-content div:\n1 2 3 4 5 6 7 8 9 10 11 12 \u0026lt;div class=\u0026#34;container body-content\u0026#34;\u0026gt; \u0026lt;div id=\u0026#34;main-content\u0026#34;\u0026gt; @RenderBody() \u0026lt;/div\u0026gt; \u0026lt;hr /\u0026gt; \u0026lt;footer\u0026gt; \u0026lt;p\u0026gt;Full-page load at @DateTime.Now\u0026lt;/p\u0026gt; \u0026lt;p\u0026gt;\u0026amp;copy; 2017 - Razor Pages\u0026lt;/p\u0026gt; \u0026lt;/footer\u0026gt; \u0026lt;/div\u0026gt; ℹ️ Note that the time of the full-page load is in the footer, so we can check that the requests are done with AJAX. Removing the layout for HTMX requests All requests done with HTMX will have a Hx-Request header set to true.\nWe will check for it in the _ViewStart.cshtml file and remove the page layout for HTMX requests:\n1 2 3 4 5 6 7 8 9 10 @{ @if (!ViewContext.HttpContext.Request.Headers[\u0026#34;Hx-Request\u0026#34;].Contains(\u0026#34;true\u0026#34;)) { Layout = \u0026#34;_Layout\u0026#34;; } else { Layout = \u0026#34;\u0026#34;; } } Setting up the forms Now, we also need to include the hx-boost and hx-target attributes in the \u0026lt;form\u0026gt; tags in the Index.cshtml, New.cshtml and Edit.cshtml files:\n\u0026lt;form method=\u0026#34;post\u0026#34; class=\u0026#34;form-horizontal\u0026#34; hx-boost=\u0026#34;true\u0026#34; hx-target=\u0026#34;#main-content\u0026#34;\u0026gt; Now we can test the application. Note that the time of the page load in the footer doesn\u0026rsquo;t change:\nLooking at the browser\u0026rsquo;s console, we can see the request was done:\nThe form content in the request payload:\nThe response without the _Layout.cshtml content:\nThe response rendered in the page:\n💡 ASP.NET Form Validation will also work: Changing the page title according to the page load Until now, the page title is not being changed, because it is set in the _Layout.cshtml file, that is loaded just on the first page load:\n\u0026lt;title\u0026gt;@ViewData[\u0026#34;Title\u0026#34;] - Razor Pages Sample + HTMX\u0026lt;/title\u0026gt; The hx-boost attribute will automatically change the page title if the response returns a \u0026lt;title\u0026gt; tag. To do this, let\u0026rsquo;s create a new layout for the HTMX requests.\nCreate a _LayoutHtmxBoost.cshtml with the below:\n1 2 3 4 5 6 7 8 9 10 @if (!string.IsNullOrEmpty(ViewData[\u0026#34;Title\u0026#34;]!.ToString())) { \u0026lt;head\u0026gt; \u0026lt;title\u0026gt;@ViewData[\u0026#34;Title\u0026#34;] - Razor Pages Sample + HTMX\u0026lt;/title\u0026gt; \u0026lt;/head\u0026gt; } @RenderBody() @RenderSection(\u0026#34;Scripts\u0026#34;, required: false) Note that we return a \u0026lt;title\u0026gt; tag if the ViewData[\u0026quot;Title\u0026quot;] has any value.\nNow, let\u0026rsquo;s change the _ViewStart.cshtml file to use the _LayoutHtmxBoost layout for HTMX requests:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 @{ @if (!ViewContext.HttpContext.Request.Headers[\u0026#34;Hx-Request\u0026#34;].Contains(\u0026#34;true\u0026#34;)) { Layout = \u0026#34;_Layout\u0026#34;; } else if (ViewContext.HttpContext.Request.Headers[\u0026#34;Hx-Boosted\u0026#34;].Contains(\u0026#34;true\u0026#34;)) { Layout = \u0026#34;_LayoutHtmxBoost\u0026#34;; } else { Layout = \u0026#34;\u0026#34;; } } ⚠️ We are checking for the Hx-Boosted header non-boosted HTMX requests don\u0026rsquo;t use this layout. 💡 HTMX automatically pushes the pages to the browser\u0026rsquo;s history: Indicating the loading status That\u0026rsquo;s cool, but we don\u0026rsquo;t have an indicator of the request being processed. For this, we can use the hx-indicator attribute, specifying the id of the element to show while awaiting for the request.\nThe hx-indicator works by adding the htmx-request css class to the element specified, that will set the opacity to 1. For this to work, we first need to hide the element with an opacity of 0, or use the htmx-indicator css class that does this with a css transaction.\nFirst, I created a partial page with a loader from Pure CSS Loaders with the name _LoadSpinner, and set the id to spinner and added the htmx-indicator css class:\n1 2 3 \u0026lt;div id=\u0026#34;spinner\u0026#34; class=\u0026#34;htmx-indicator\u0026#34;\u0026gt; \u0026lt;div class=\u0026#34;lds-ring\u0026#34;\u0026gt;\u0026lt;div\u0026gt;\u0026lt;/div\u0026gt;\u0026lt;div\u0026gt;\u0026lt;/div\u0026gt;\u0026lt;div\u0026gt;\u0026lt;/div\u0026gt;\u0026lt;div\u0026gt;\u0026lt;/div\u0026gt;\u0026lt;/div\u0026gt; \u0026lt;/div\u0026gt; Then, I added it in the menu:\n1 2 3 4 5 6 7 \u0026lt;div class=\u0026#34;collapse navbar-collapse\u0026#34; id=\u0026#34;header-navbar-collapse\u0026#34;\u0026gt; \u0026lt;ul class=\u0026#34;nav navbar-nav\u0026#34;\u0026gt; \u0026lt;li\u0026gt;\u0026lt;a asp-page=\u0026#34;/Index\u0026#34;\u0026gt;Home\u0026lt;/a\u0026gt;\u0026lt;/li\u0026gt; \u0026lt;li\u0026gt;\u0026lt;a asp-page=\u0026#34;/Customers/Index\u0026#34;\u0026gt;Customers\u0026lt;/a\u0026gt;\u0026lt;/li\u0026gt; \u0026lt;li\u0026gt;\u0026lt;partial name=\u0026#34;_LoadSpinner\u0026#34; /\u0026gt;\u0026lt;/li\u0026gt; \u0026lt;/ul\u0026gt; \u0026lt;/div\u0026gt; Lastly, I add the hx-indicator attribute with the value #spinner to the \u0026lt;nav\u0026gt; element:\n\u0026lt;nav class=\u0026#34;navbar navbar-inverse navbar-fixed-top\u0026#34; hx-boost=\u0026#34;true\u0026#34; hx-target=\u0026#34;#main-content\u0026#34; hx-indicator=\u0026#34;#spinner\u0026#34;\u0026gt; and all \u0026lt;form\u0026gt; elements:\n\u0026lt;form method=\u0026#34;post\u0026#34; class=\u0026#34;form-horizontal\u0026#34; hx-boost=\u0026#34;true\u0026#34; hx-target=\u0026#34;#main-content\u0026#34; hx-indicator=\u0026#34;#spinner\u0026#34;\u0026gt; 💡 To make it easier to see the indicator, I included a delay of 500 milliseconds to all pages GET/POST methods: 1 2 3 4 5 6 public async Task\u0026lt;IActionResult\u0026gt; OnGetAsync(int id) { await Task.Delay(TimeSpan.FromMilliseconds(500)); ... } Running the app again, we can see the indicator showing in every actions:\nFull source code GitHub Repository\nReferences and Links Razor Pages Sample by Damian Edwards hx-boost hx-target hx-indicator ","date":"2023-10-11T08:20:00-03:00","dateFormatted":"11 Oct 2023","image":"https://genezini.com/blog/p/server-side-rendered-spas-with-asp.net-and-no-javascript/cover.jpg","permalink":"https://genezini.com/blog/p/server-side-rendered-spas-with-asp.net-and-no-javascript/","title":"Server-side rendered SPAs with ASP.NET and no Javascript"},{"content":"Introduction As I explained previously in this article, in integration tests, we should mock unmanaged dependencies (dependencies that are external to our system and not controlled by us, like APIs) but test against real managed dependencies (dependencies that are controlled by our system, like databases, queues, etc). This improves the reliability of the integration tests because the communication with these dependencies are a complex part of the system and can break with a package update, a database update or a change in a queue message format.\nIn this post, I\u0026rsquo;ll show how to use Localstack and Testcontainers to emulate an AWS environment for use in integration tests.\nWhy use Localstack for integration tests? Reduced Costs: Using LocalStack eliminates the use of AWS resources during tests and development. It also avoids accidental charges during development, for example, in cases of incorrect logic; Development speed: Using LocalStack allows developer to test without having to deploy or configure AWS credentials on the local environment. It also remove external factors, as other people using the same AWS resources on the environment; Reproducibility and less flaky tests: Integration tests run in an isolated environment, avoiding any interference with the production or staging environment. This makes tests reproducible in any developer\u0026rsquo;s machine and less flaky because there is no dependency with the network and a shared AWS environment, that can change frequently; Cleanup: LocalStack environment is automatically cleaned up after the tests, making easier to use in automated tests; Test of Edge Cases: We can customize LocalStack\u0026rsquo;s configuration, allowing us to test edge cases that may not be tested on an AWS environment, like rate limits and permissions errors. What is Testcontainers? Testcontainers is a library that manages containers\u0026rsquo; lifecycle to be used in automated tests. These containers are useful for testing applications against real managed dependencies, like databases, or AWS resources (using LocalStack) that can be created and disposed of after the tests.\nRunning a disposable container with Testcontainers To use Testcontainers, you will need to have a container runtime (Docker, Podman, Rancher, etc) installed on your machine.\nThen, you need to add the Testcontainers NuGet package to your test project:\ndotnet add package Testcontainers To run a LocalStack container, we first need to use the ContainerBuilder class to build an IContainer:\n1 2 3 4 5 6 7 8 9 10 11 12 const int LocalStackPort = 4566; const string LocalStackImage = \u0026#34;localstack/localstack:1.3.1\u0026#34;; await using var LocalStackTestcontainer = new ContainerBuilder() .WithImage(LocalStackImage) .WithExposedPort(LocalStackPort) .WithPortBinding(LocalStackPort, true) .WithWaitStrategy(Wait.ForUnixContainer() .UntilHttpRequestIsSucceeded(request =\u0026gt; request .ForPath(\u0026#34;/_localstack/health\u0026#34;) .ForPort(LocalStackPort))) .Build(); Then, we start the container and use the Hostname property and GetMappedPublicPort() method to create the ServiceUrl that will be used by the AWS Clients:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 const int LocalStackPort = 4566; const string LocalStackImage = \u0026#34;localstack/localstack:1.3.1\u0026#34;; await using var LocalStackTestcontainer = new ContainerBuilder() .WithImage(LocalStackImage) .WithExposedPort(LocalStackPort) .WithPortBinding(LocalStackPort, true) .WithWaitStrategy(Wait.ForUnixContainer() .UntilHttpRequestIsSucceeded(request =\u0026gt; request .ForPath(\u0026#34;/_localstack/health\u0026#34;) .ForPort(LocalStackPort))) .Build(); await LocalStackTestcontainer.StartAsync(); var localstackUrl = $\u0026#34;http://{_localStackTestcontainer.Hostname}:{_localStackTestcontainer.GetMappedPublicPort(4566)}\u0026#34;; 🚨 The IContainer interface extends IAsyncDisposable and needs to be disposed of after use. We can use the await using syntax, as in the example above, or call the DisposeAsync method, as in the fixture shown below. Example context In the example from Part 1, I have a controller with a POST method that uploads an image to an S3 bucket and a GET method that finds an image from the S3 bucket by its file name:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 app.MapPost(\u0026#34;/upload\u0026#34;, async (IAmazonS3 s3Client, IFormFile file) =\u0026gt; { var bucketName = builder.Configuration[\u0026#34;BucketName\u0026#34;]!; var bucketExists = await s3Client.DoesS3BucketExistAsync(bucketName); if (!bucketExists) { return Results.BadRequest($\u0026#34;Bucket {bucketName} does not exists.\u0026#34;); } using var fileStream = file.OpenReadStream(); var putObjectRequest = new PutObjectRequest() { BucketName = bucketName, Key = file.FileName, InputStream = fileStream }; putObjectRequest.Metadata.Add(\u0026#34;Content-Type\u0026#34;, file.ContentType); var putResult = await s3Client.PutObjectAsync(putObjectRequest); return Results.Ok($\u0026#34;File {file.FileName} uploaded to S3 successfully!\u0026#34;); }); app.MapGet(\u0026#34;/object/{key}\u0026#34;, async (IAmazonS3 s3Client, string key) =\u0026gt; { var bucketName = builder.Configuration[\u0026#34;BucketName\u0026#34;]!; var bucketExists = await s3Client.DoesS3BucketExistAsync(bucketName); if (!bucketExists) { return Results.BadRequest($\u0026#34;Bucket {bucketName} does not exists.\u0026#34;); } try { var getObjectResponse = await s3Client.GetObjectAsync(bucketName, key); return Results.File(getObjectResponse.ResponseStream, getObjectResponse.Headers.ContentType); } catch (AmazonS3Exception ex) when (ex.ErrorCode.Equals(\u0026#34;NotFound\u0026#34;, StringComparison.OrdinalIgnoreCase)) { return Results.NotFound(); } }); ⚠️ The business logic in the controller is just for the sake of simplicity. In a real-world application, the logic should be in a Use Case/Interactor or something with the same purpose. Creating integration tests with Testcontainers In this example, I\u0026rsquo;m using xUnit and the WebApplicationFactory\u0026lt;T\u0026gt; class from ASP.NET Core.\nIf you don\u0026rsquo;t know how to use the WebApplicationFactory\u0026lt;T\u0026gt; class, I explained in this post.\nWhen running in AWS, the AmazonS3Client will get its access data from the IAM Role attached to the service running it. When running locally, it will get from the AWS CLI profile named default or from the settings we pass to it.\nℹ️ The default profile is read from the AWSCLI credentials file (%userprofile%\\.aws\\credentials on Windows and ~/.aws/credentials on Linux). In the code below, I\u0026rsquo;m checking for a configuration section with the name AWS, that is not present in the production environment. If it\u0026rsquo;s found, I set the Region, ServiceURL and ForcePathStyle properties of the AmazonS3Config and pass it to the creation of the AmazonS3Client:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 builder.Services.AddSingleton\u0026lt;IAmazonS3\u0026gt;(sc =\u0026gt; { var configuration = sc.GetRequiredService\u0026lt;IConfiguration\u0026gt;(); var awsConfiguration = configuration.GetSection(\u0026#34;AWS\u0026#34;).Get\u0026lt;AwsConfiguration\u0026gt;(); if (awsConfiguration?.ServiceURL is null) { return new AmazonS3Client(); } else { return AwsS3ClientFactory.CreateAwsS3Client( awsConfiguration.ServiceURL, awsConfiguration.Region, awsConfiguration.ForcePathStyle, awsConfiguration.AwsAccessKey, awsConfiguration.AwsSecretKey); } }); For the integration tests, those settings (except for the ServiceURL) will be configured in appsettings.IntegrationTest.json file, that is injected by the WebApplicationFactory\u0026lt;T\u0026gt; class:\n1 2 3 4 5 6 7 8 9 10 { ... \u0026#34;AWS\u0026#34;: { \u0026#34;Region\u0026#34;: \u0026#34;us-east-1\u0026#34;, \u0026#34;ForcePathStyle\u0026#34;: \u0026#34;true\u0026#34;, \u0026#34;AwsAccessKey\u0026#34;: \u0026#34;test\u0026#34;, \u0026#34;AwsSecretKey\u0026#34;: \u0026#34;test\u0026#34; } } ℹ️ As the container will have a random public port, the AWS:ServiceURL configuration has to be passed after Testcontainers starts the LocalStack container, using the Hostname and GetMappedPublicPort(4566) to create the full URL. LocalStack/Testcontainer Fixture This fixture will start and dispose of the Localstack container, sharing it with all tests.\nI recommend using the same container instance for all tests, instead of one per test. It will require more attention to avoid cause between the tests, but will save time in executing them.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 [CollectionDefinition(\u0026#34;LocalStackTestcontainer Collection\u0026#34;)] public class LocalStackTestcontainerCollection : ICollectionFixture\u0026lt;LocalStackTestcontainerFixture\u0026gt; { } public class LocalStackTestcontainerFixture : IAsyncLifetime { public const int LocalStackPort = 4566; public const string LocalStackImage = \u0026#34;localstack/localstack:1.3.1\u0026#34;; public IContainer LocalStackTestcontainer { get; private set; } = default!; public async Task InitializeAsync() { LocalStackTestcontainer = new ContainerBuilder() .WithImage(LocalStackImage) .WithExposedPort(LocalStackPort) .WithPortBinding(LocalStackPort, true) .WithWaitStrategy(Wait.ForUnixContainer() .UntilHttpRequestIsSucceeded(request =\u0026gt; request .ForPath(\u0026#34;/_localstack/health\u0026#34;) .ForPort(LocalStackPort))) .Build(); await LocalStackTestcontainer.StartAsync(); } public async Task DisposeAsync() { await LocalStackTestcontainer.DisposeAsync(); } } Upload an image test The test reads the configurations with IConfiguration to create an AmazonS3Client and create the bucket in the LocalStack environment and to assert the test condition (the object was created in the bucket):\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 [Collection(\u0026#34;LocalStackTestcontainer Collection\u0026#34;)] public class UploadTests : IClassFixture\u0026lt;CustomWebApplicationFactory\u0026gt; { private readonly CustomWebApplicationFactory _factory; private readonly IContainer _localStackTestcontainer; public UploadTests(CustomWebApplicationFactory factory, LocalStackTestcontainerFixture localStackTestcontainerFixture) { _factory = factory; _localStackTestcontainer = localStackTestcontainerFixture.LocalStackTestcontainer; } [Fact] public async Task UploadObject_Returns200() { //Arrange var localstackUrl = $\u0026#34;http://{_localStackTestcontainer.Hostname}:{_localStackTestcontainer.GetMappedPublicPort(4566)}\u0026#34;; var HttpClient = _factory .WithWebHostBuilder(builder =\u0026gt; { builder.UseSetting(\u0026#34;AWS:ServiceURL\u0026#34;, localstackUrl); }) .CreateClient(); var configuration = _factory.Services.GetRequiredService\u0026lt;IConfiguration\u0026gt;(); var awsConfiguration = configuration.GetSection(\u0026#34;AWS\u0026#34;).Get\u0026lt;AwsConfiguration\u0026gt;()!; var s3Client = AwsS3ClientFactory.CreateAwsS3Client(localstackUrl, awsConfiguration.Region, awsConfiguration.ForcePathStyle, awsConfiguration.AwsAccessKey, awsConfiguration.AwsSecretKey); await s3Client.PutBucketAsync(configuration[\u0026#34;BucketName\u0026#34;]); const string fileName = \u0026#34;upload.jpg\u0026#34;; var filePath = Path.Combine(Directory.GetCurrentDirectory(), \u0026#34;Assets\u0026#34;, fileName); //Act using var multipartFormContent = new MultipartFormDataContent(); var fileStreamContent = new StreamContent(File.OpenRead(filePath)); fileStreamContent.Headers.ContentType = new MediaTypeHeaderValue(\u0026#34;image/jpg\u0026#34;); multipartFormContent.Add(fileStreamContent, name: \u0026#34;file\u0026#34;, fileName: fileName); var httpResponse = await HttpClient.PostAsync($\u0026#34;/upload\u0026#34;, multipartFormContent); //Assert httpResponse.StatusCode.Should().Be(HttpStatusCode.OK); var bucketFile = await s3Client.GetObjectAsync(configuration[\u0026#34;BucketName\u0026#34;], fileName); bucketFile.HttpStatusCode.Should().Be(HttpStatusCode.OK); } } ℹ️ We could also create the S3 bucket using the ExecAsync method from Testcontainer\u0026rsquo;s IContainer to execute AWS CLI commands in the Localstack container, but I find it easier and less error prone to do it with the AWS SDK. Upload an existing image test This test uses the ListObjectsAsync method of the AmazonS3Client to assert that uploading an image with the same file name will override the image instead of creating another:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 [Fact] public async Task UploadExistentObject_Returns200AndOverride() { //Arrange var localstackUrl = $\u0026#34;http://{_localStackTestcontainer.Hostname}:{_localStackTestcontainer.GetMappedPublicPort(4566)}\u0026#34;; var HttpClient = _factory .WithWebHostBuilder(builder =\u0026gt; { builder.UseSetting(\u0026#34;AWS:ServiceURL\u0026#34;, localstackUrl); }) .CreateClient(); var configuration = _factory.Services.GetRequiredService\u0026lt;IConfiguration\u0026gt;(); var awsConfiguration = configuration.GetSection(\u0026#34;AWS\u0026#34;).Get\u0026lt;AwsConfiguration\u0026gt;()!; var s3Client = AwsS3ClientFactory.CreateAwsS3Client(localstackUrl, awsConfiguration.Region, awsConfiguration.ForcePathStyle, awsConfiguration.AwsAccessKey, awsConfiguration.AwsSecretKey); await s3Client.PutBucketAsync(configuration[\u0026#34;BucketName\u0026#34;]); const string fileName = \u0026#34;upload.jpg\u0026#34;; var filePath = Path.Combine(Directory.GetCurrentDirectory(), \u0026#34;Assets\u0026#34;, fileName); //Act using var multipartFormContent = new MultipartFormDataContent(); var fileStreamContent = new StreamContent(File.OpenRead(filePath)); fileStreamContent.Headers.ContentType = new MediaTypeHeaderValue(\u0026#34;image/jpg\u0026#34;); multipartFormContent.Add(fileStreamContent, name: \u0026#34;file\u0026#34;, fileName: fileName); var httpResponse1 = await HttpClient.PostAsync($\u0026#34;/upload\u0026#34;, multipartFormContent); var httpResponse2 = await HttpClient.PostAsync($\u0026#34;/upload\u0026#34;, multipartFormContent); //Assert httpResponse1.StatusCode.Should().Be(HttpStatusCode.OK); httpResponse2.StatusCode.Should().Be(HttpStatusCode.OK); var bucketObjects = await s3Client.ListObjectsAsync(configuration[\u0026#34;BucketName\u0026#34;]); bucketObjects.S3Objects.Count.Should().Be(1); var bucketFile = await s3Client.GetObjectAsync(configuration[\u0026#34;BucketName\u0026#34;], fileName); bucketFile.HttpStatusCode.Should().Be(HttpStatusCode.OK); } Get an image test To test the endpoint that returns the image, I use the AmazonS3Client to create the bucket and upload an image, and then, call the endpoint and assert that it returns the image:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 [Collection(\u0026#34;LocalStackTestcontainer Collection\u0026#34;)] public class GetObjectTests : IClassFixture\u0026lt;CustomWebApplicationFactory\u0026gt; { private readonly CustomWebApplicationFactory _factory; private readonly IContainer _localStackTestcontainer; public GetObjectTests(CustomWebApplicationFactory factory, LocalStackTestcontainerFixture localStackTestcontainerFixture) { _factory = factory; _localStackTestcontainer = localStackTestcontainerFixture.LocalStackTestcontainer; } [Fact] public async Task GetExistingObject_Returns200() { //Arrange var localstackUrl = $\u0026#34;http://{_localStackTestcontainer.Hostname}:{_localStackTestcontainer.GetMappedPublicPort(4566)}\u0026#34;; var HttpClient = _factory .WithWebHostBuilder(builder =\u0026gt; { builder.UseSetting(\u0026#34;AWS:ServiceURL\u0026#34;, localstackUrl); }) .CreateClient(); var configuration = _factory.Services.GetRequiredService\u0026lt;IConfiguration\u0026gt;(); var awsConfiguration = configuration.GetSection(\u0026#34;AWS\u0026#34;).Get\u0026lt;AwsConfiguration\u0026gt;()!; var s3Client = AwsS3ClientFactory.CreateAwsS3Client(localstackUrl, awsConfiguration.Region, awsConfiguration.ForcePathStyle, awsConfiguration.AwsAccessKey, awsConfiguration.AwsSecretKey); await s3Client.PutBucketAsync(configuration[\u0026#34;BucketName\u0026#34;]); const string fileName = \u0026#34;upload.jpg\u0026#34;; var filePath = Path.Combine(Directory.GetCurrentDirectory(), \u0026#34;Assets\u0026#34;, fileName); var putObjectRequest = new PutObjectRequest() { BucketName = configuration[\u0026#34;BucketName\u0026#34;], Key = fileName, FilePath = filePath }; putObjectRequest.Metadata.Add(\u0026#34;Content-Type\u0026#34;, \u0026#34;image/jpg\u0026#34;); var putResult = await s3Client.PutObjectAsync(putObjectRequest); //Act var httpResponse = await HttpClient.GetAsync($\u0026#34;/object/{fileName}\u0026#34;); //Assert httpResponse.StatusCode.Should().Be(HttpStatusCode.OK); httpResponse.Content.Headers.ContentType.Should().Be(MediaTypeHeaderValue.Parse(\u0026#34;image/jpeg\u0026#34;)); } } Image not found test To test the behavior when the image doesn\u0026rsquo;t exist, I use the AmazonS3Client to create the bucket, and then, call the endpoint that returns the image with a file name that doesn\u0026rsquo;t exist in the bucket:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 [Fact] public async Task GetInexistingObject_Returns404() { //Arrange var localstackUrl = $\u0026#34;http://{_localStackTestcontainer.Hostname}:{_localStackTestcontainer.GetMappedPublicPort(4566)}\u0026#34;; var HttpClient = _factory .WithWebHostBuilder(builder =\u0026gt; { builder.UseSetting(\u0026#34;AWS:ServiceURL\u0026#34;, localstackUrl); }) .CreateClient(); var configuration = _factory.Services.GetRequiredService\u0026lt;IConfiguration\u0026gt;(); var awsConfiguration = configuration.GetSection(\u0026#34;AWS\u0026#34;).Get\u0026lt;AwsConfiguration\u0026gt;()!; var s3Client = AwsS3ClientFactory.CreateAwsS3Client(localstackUrl, awsConfiguration.Region, awsConfiguration.ForcePathStyle, awsConfiguration.AwsAccessKey, awsConfiguration.AwsSecretKey); await s3Client.PutBucketAsync(configuration[\u0026#34;BucketName\u0026#34;]); const string fileName = \u0026#34;inexisting.jpg\u0026#34;; //Act var httpResponse = await HttpClient.GetAsync($\u0026#34;/object/{fileName}\u0026#34;); //Assert httpResponse.StatusCode.Should().Be(HttpStatusCode.NotFound); } Full source code GitHub repository\nReferences and Links LocalStack Website LocalStack Feature Coverage Testcontainers ","date":"2023-07-31T20:00:00-03:00","dateFormatted":"31 Jul 2023","image":"https://genezini.com/blog/p/integration-tests-with-aws-s3-buckets-using-localstack-and-testcontainers/cover.jpg","permalink":"https://genezini.com/blog/p/integration-tests-with-aws-s3-buckets-using-localstack-and-testcontainers/","title":"Integration tests with AWS S3 buckets using Localstack and Testcontainers"},{"content":"Introduction Static code analyzers are tools used to analyze software code without executing it. They can examine the code to find code smells, vulnerabilities, potential errors, and code out of a defined standard, for example. They work by parsing the source code and evaluating its syntax (structure of the code) and semantic (meaning of the code).\nRoslyn, the C# compiler, provides tools for developing Roslyn Analyzers (Static code analyzers for Roslyn), giving access to the syntax and semantic of the code, that can be executed at development and build time, providing feedback in near real-time to the developers.\nIn this post, I\u0026rsquo;ll show a Roslyn Analyzer I built for using Discriminated Unions in C#, requiring to check the union type before access, and talk about some lessons learned while developing it.\nDiscriminatedUnions.Net package The package allows the use of Discriminated Unions by enforcing two rules:\nThe type of a Union must be checked before access; All types of a Union must be checked (or have an else/default/discard case). To use the package, install it in the project:\n\u0026lt;PackageReference Include=\u0026#34;DiscriminatedUnions.Net\u0026#34; Version=\u0026#34;1.0.0.19\u0026#34; /\u0026gt; Extend UnionValue when declaring the types that will be used in the Discriminated Union:\n1 2 3 4 5 6 7 public class Bird: UnionValue { } public class Dog : UnionValue { } Declare the Union member passing the possible value types:\nUnion\u0026lt;Dog, Bird\u0026gt; animal = new Dog(); And access it checking for the type with If, Switch/Case or Pattern Matching:\n1 2 3 4 5 6 if (animal.Value is Dog) { Console.WriteLine($\u0026#34;Dog: {animal.Value}\u0026#34;); } else ... Accessing the object without checking will yield an error DUN002 - 'animal.Value' not checked before access:\nNot checking for all possible types will yield an error DUN001 - 'animal' not being evaluated for all possible types:\n⚠️ This package was made for study purposes only. Feel free to test it, but won\u0026rsquo;t use it for production code as it won\u0026rsquo;t be maintained and may not cover all edge cases. DiscriminatedUnions.NET Source Code https://github.com/dgenezini/DiscriminatedUnions.NET\nLessons learned Development-time only nuget packages Roslyn analyzers can be distributed as nuget packages, but it is important to generate the package as a Development Dependency only, setting the property DevelopmentDependency as true on the csproj file:\n\u0026lt;DevelopmentDependency\u0026gt;true\u0026lt;/DevelopmentDependency\u0026gt; This will generate two tags in the nuget package properties of the consumer projects:\n1 2 3 4 \u0026lt;PackageReference Include=\u0026#34;DiscriminatedUnions.Net.Analyzers\u0026#34; Version=\u0026#34;1.0.0.1\u0026#34;\u0026gt; \u0026lt;PrivateAssets\u0026gt;all\u0026lt;/PrivateAssets\u0026gt; \u0026lt;IncludeAssets\u0026gt;runtime; build; native; contentfiles; analyzers; buildtransitive\u0026lt;/IncludeAssets\u0026gt; \u0026lt;/PackageReference\u0026gt; PrivateAssets with the value all, indicating that this package won\u0026rsquo;t flow to projects depending on the consumer project. For example, in a scenario where Project1 depends on the DiscriminatedUnions.Net.Analyzers package and Project2 depends on Project1, the analyzer won\u0026rsquo;t be available for Project2; IncludeAssets without the compile value in its value, indicating the consumer of the package won\u0026rsquo;t have access to its compiled assemblies. Roslyn Analyzers can be used to enforce rules on how to use a package A great use case for Roslyn Analyzers is to enforce rules on nuget packages.\nThe DistributedUnion.Net package is one example. It has the Union and UnionValue types and analyzers to enforce the rules on its use.\nBecause classes inside Analyzers packages are not accessible by consumers (and shouldn\u0026rsquo;t be), the most correct way, in my opinion, is to have the public classes in one package and this package will consume another package with the analyzers, but removing the PrivateAssets configuration. This way, the analyzers will be available for consumers of the parent package:\n1 2 3 \u0026lt;PackageReference Include=\u0026#34;DiscriminatedUnions.Net.Analyzers\u0026#34; Version=\u0026#34;0.1.0.0-beta\u0026#34;\u0026gt; \u0026lt;IncludeAssets\u0026gt;runtime; build; native; contentfiles; analyzers; buildtransitive\u0026lt;/IncludeAssets\u0026gt; \u0026lt;/PackageReference\u0026gt; Analyzers have to evolve together with the language Because analyzers use the code syntax, as new ways of doing something are added to the language, analyzers have to be updated to treat them also. For example:\nAn analyzer checking for conditions before pattern matching was release, would need to be updated to consider this new syntax; An analyzer using types for its rules, would need to be updated to consider nullable reference types in its logic. TDD is your friend The Roslyn Analyzers project template comes with a VSIX project that can be run to test the analyzers, but I couldn\u0026rsquo;t get them to work and decided it wasn\u0026rsquo;t worth the time.\nTurns out the best and easiest way to run and experiment with the analyzer is by running automated tests. Create the test, run the test and change the analyzer until the test passes.\nExample of an automated test that expects the error code DUN001 at the location 0 with the argument unionBird:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 [TestMethod] public async Task Notify_If_NotCheckingAllCases_Many() { var test = @\u0026#34; using DiscriminatedUnionsNet; class Program { static string GetValue() { Union\u0026lt;Duck, Goose, Eagle, Crow\u0026gt; unionBird = new Duck(); if ({|#0:unionBird|}.Value is Goose) { return \u0026#34;\u0026#34;Goose\u0026#34;\u0026#34;; } return null; } }\u0026#34; + defineUnion; var expected = VerifyCS .Diagnostic(\u0026#34;DUN001\u0026#34;) .WithLocation(0) .WithArguments(\u0026#34;unionBird\u0026#34;); await VerifyCS.VerifyAnalyzerAsync(test, expected); } The location and argument are marked with the {|#location:argument|} syntax.\nℹ️ The project template comes with some basic tests as examples that are easy to change. Casts\u0026hellip; Casts everywhere The analyzers methods and properties define general interface types that need to be cast to the specific type before accessing its properties.\nThis is an excerpt from the DiscriminatedUnions.Net package:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 var namedType = (INamedTypeSymbol)typeInfo.Type; //typeInfo.Type is ITypeSymbol if ((!namedType.Name.Equals(\u0026#34;UnionValue\u0026#34;)) || (!namedType.ContainingNamespace.Name.Equals(\u0026#34;DiscriminatedUnionsNet\u0026#34;))) { return; } ... if (parent is ISwitchOperation switchOperation) //parent is IOperation { if (!(switchOperation.Value is IPropertyReferenceOperation propertyReferenceOperation) || //switchOperation.Value is IOperation !(propertyReferenceOperation.Instance is ILocalReferenceOperation parentReferenceOperation)) { parent = parent.Parent; continue; } ... } else if (parent is ISwitchExpressionOperation switchExpressionOperation) //parent is IOperation { ... } Understanding the syntax tree and finding the correct interfaces The easiest way to understand the syntax tree is by using the Visual Studio\u0026rsquo;s Syntax Visualizer (Installation instructions).\nJust click anywhere in the code and it will show the syntax tree up to that point:\nAnother tip is to type ISymbol, IOperation, or any base interface to see all the specific interfaces in the intellisense:\nNot many material to help beginners Because Roslyn Analyzers have very specific use cases, there is not a lot of materials and documentation online. Here are some links that helped me learn (specially Josh Varty\u0026rsquo;s and Meziantou\u0026rsquo;s Blog):\nJosh Varty\u0026rsquo;s Blog - Learn Roslyn Now series\nMeziantou\u0026rsquo;s Blog - Writing a Roslyn analyzer series\nRoslyn docs - How to start\nRoslyn GitHub docs - How to write a C# Analyzer and Code Fix\nRoslyn GitHub docs - Analyzer Actions Semantics\n","date":"2023-07-12T20:15:00-03:00","dateFormatted":"12 Jul 2023","image":"https://genezini.com/blog/p/lessons-learned-from-building-a-static-code-analyzer-for-csharp/cover.jpg","permalink":"https://genezini.com/blog/p/lessons-learned-from-building-a-static-code-analyzer-for-csharp/","title":"Lessons learned from building a static code analyzer for C#"},{"content":"Introduction Blazor was initially developed to build interactive web applications using C#, but can now be used in many platforms when used together with .NET MAUI.\nIn this post, I\u0026rsquo;ll show how to use Blazor Hybrid to develop a Who\u0026rsquo;s that Pokémon game for Android.\nWhat is MAUI? MAUI (Multi-platform App UI) is a cross-platform framework to create native mobile apps and desktop apps with .NET.\nUsing MAUI, we can create apps that run on multiple platforms (Windows, MacOS, Android and iOS) with the same codebase (and some specific code for each platform, if necessary).\nWhat is Blazor Hybrid? Blazor Hybrid allows us to run Blazor components natively in mobile and desktop apps using a Web View with full access to the devices capabilities. There is no need for a server to render the components and WebAssembly isn\u0026rsquo;t involved.\nInstalling MAUI MAUI doesn\u0026rsquo;t come with Visual Studio or the .NET SDK by default. We need to install its workload manually using the dotnet workload install command:\ndotnet workload install maui ℹ️ The installation process may take some time to complete. After installing, the MAUI templates will show in Visual Studio. We\u0026rsquo;ll use the .NET MAUI Blazor Hybrid App.\nThe components The app is composed mainly of the four components marked in the image and described below.\nWhosThatPokemon: Renders the other components + logic to select a random pokemon and check if Pokémon selected by the user is correct; WhosThatPokemonData: Renders the Pokémon image and name; PokemonSelector: Component to search and select a Pokémon; WhosThatPokemonResult: Show if the selected Pokémon was the right one. Creating a service to store the Pokémon information First, I created a service to get the list of Pokémon from a public API (PokéAPI) using Refit and cache it locally to be used by any component. This information is static, so there is no need to go to the API every time.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 using BlazorHybridApp.Components.Domain; using BlazorHybridApp.Components.Interfaces.APIEndpoints; using Microsoft.Extensions.Configuration; using Refit; namespace BlazorHybridApp.Components.Services; public class PokemonDataService { private List\u0026lt;PokemonInfo\u0026gt;? _pokemonList; private readonly IPokeApi _pokeApi; public PokemonDataService(IConfiguration configuration) { _pokeApi ??= RestService.For\u0026lt;IPokeApi\u0026gt;(configuration![\u0026#34;PokeApiBaseUrl\u0026#34;]!); } public async Task\u0026lt;List\u0026lt;PokemonInfo\u0026gt;\u0026gt; GetPokemonListAsync() { if (_pokemonList == null) { var species = await _pokeApi.GetAllPokemonSpecies(); _pokemonList = species.Results .Select(pokemonSpecie =\u0026gt; new PokemonInfo() { Id = GetPokemonIdFromUrl(pokemonSpecie.Url), Name = pokemonSpecie.Name.ToUpperInvariant() }) .ToList(); } return _pokemonList; } public async Task\u0026lt;PokemonInfo\u0026gt; GetPokemonByIdAsync(int id) { _pokemonList ??= await GetPokemonListAsync(); return _pokemonList.Single(a =\u0026gt; a.Id == id); } public async Task\u0026lt;int\u0026gt; GetPokemonCountAsync() { _pokemonList ??= await GetPokemonListAsync(); return _pokemonList.Count; } private static int GetPokemonIdFromUrl(string url) { return int.Parse(new Uri(url).Segments.LastOrDefault()?.Trim(\u0026#39;/\u0026#39;) ?? \u0026#34;0\u0026#34;); } } And added it as a singleton in the dependency injection container:\nbuilder.Services.AddSingleton\u0026lt;PokemonDataService\u0026gt;(); Installing MudBlazor components The app uses the MudBlazor library, a free open-source component library for Blazor build entirely with C# (not Javascript wrapped components). The list of components and how to use them can be seen here.\nInstructions on how to install may change for newer versions, so I\u0026rsquo;ll link the official docs.\nConfiguring the app layout The Blazor Hybrid app template comes with a MAUI app rendering an Index.razor component page.\nLet\u0026rsquo;s edit it to render the WhosThatPokemon component we\u0026rsquo;ll create:\n1 2 3 @page \u0026#34;/\u0026#34; \u0026lt;WhosThatPokemon\u0026gt;\u0026lt;/WhosThatPokemon\u0026gt; Then, let\u0026rsquo;s modify the MainLayout.razor removing the menu:\n1 2 3 4 5 6 7 8 9 @inherits LayoutComponentBase \u0026lt;MudThemeProvider /\u0026gt; \u0026lt;div class=\u0026#34;page\u0026#34;\u0026gt; \u0026lt;main\u0026gt; @Body \u0026lt;/main\u0026gt; \u0026lt;/div\u0026gt; And add some customizations in the app.css:\n1 2 3 4 5 body { overflow-y: scroll; background-color: red; } ... Creating the components Now, let\u0026rsquo;s create the components in a Components folder.\nWhosThatPokemonData WhosThatPokemonData.razor Here we check the Show variable to decide if we should show the pokemon image and name.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 \u0026lt;div class=\u0026#34;container\u0026#34;\u0026gt; @if (!Show) { \u0026lt;div\u0026gt; \u0026lt;img src=\u0026#34;https://raw.githubusercontent.com/PokeAPI/sprites/master/sprites/pokemon/other/official-artwork/@(PokemonId).png\u0026#34; class=\u0026#34;pokemon-image pokemon-image-hidden\u0026#34; /\u0026gt; \u0026lt;/div\u0026gt; \u0026lt;div class=\u0026#34;pokemon-name\u0026#34;\u0026gt; \u0026lt;p\u0026gt;?????\u0026lt;/p\u0026gt; \u0026lt;/div\u0026gt; } else { \u0026lt;div\u0026gt; \u0026lt;img src=\u0026#34;https://raw.githubusercontent.com/PokeAPI/sprites/master/sprites/pokemon/other/official-artwork/@(PokemonId).png\u0026#34; class=\u0026#34;pokemon-image\u0026#34; /\u0026gt; \u0026lt;/div\u0026gt; \u0026lt;div class=\u0026#34;pokemon-name\u0026#34;\u0026gt; \u0026lt;p\u0026gt;@PokemonName (#@PokemonId)\u0026lt;/p\u0026gt; \u0026lt;/div\u0026gt; } \u0026lt;/div\u0026gt; WhosThatPokemonData.razor.css Here we have the \u0026ldquo;magic\u0026rdquo; to hide the Pokémon image using filter: brightness(0).\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 .container { font-family: Verdana; color: black; } .pokemon-image { width: 200px; height: 200px; filter: drop-shadow(2px 4px 6px black); margin: auto; display: block; } .pokemon-image-hidden { filter: brightness(0) drop-shadow(2px 4px 6px black) } .pokemon-name { font-size: large; } WhosThatPokemonData.razor.cs No logic, just parameters used to render the component.\n1 2 3 4 5 6 7 8 9 10 11 12 13 using Microsoft.AspNetCore.Components; namespace BlazorHybridApp.Components; public partial class WhosThatPokemonData { [Parameter] public int PokemonId { get; init; } [Parameter] public required string PokemonName { get; init; } [Parameter] public bool Show { get; init; } } PokemonSelector PokemonSelector.razor I used the MudAutocomplete component from MudBlazor. The value selected is binded to the SelectedPokemon property.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 @using BlazorHybridApp.Components.Domain; \u0026lt;div class=\u0026#34;container\u0026#34;\u0026gt; @if (_pokemon == null) { \u0026lt;MudSkeleton SkeletonType=\u0026#34;SkeletonType.Rectangle\u0026#34; Width=\u0026#34;100%\u0026#34; Height=\u0026#34;30px\u0026#34; /\u0026gt; } else { \u0026lt;MudAutocomplete @ref=\u0026#34;_selectedPokemon\u0026#34; T=\u0026#34;PokemonInfo\u0026#34; @bind-Value=\u0026#34;SelectedPokemon\u0026#34; ResetValueOnEmptyText=\u0026#34;true\u0026#34; Label=\u0026#34;Who\u0026#39;s that pokémon?\u0026#34; ToStringFunc=\u0026#34;@(e =\u0026gt; e?.Name)\u0026#34; SearchFunc=\u0026#34;@Search\u0026#34; /\u0026gt; } \u0026lt;/div\u0026gt; PokemonSelector.razor.cs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 using BlazorHybridApp.Components.Domain; using BlazorHybridApp.Components.Services; using Microsoft.AspNetCore.Components; using MudBlazor; namespace BlazorHybridApp.Components; public partial class PokemonSelector { [Inject] private PokemonDataService PokemonDataService { get; init; } = default!; private MudAutocomplete\u0026lt;PokemonInfo\u0026gt; _selectedPokemon = default!; private List\u0026lt;PokemonInfo\u0026gt;? _pokemon; public PokemonInfo? SelectedPokemon { get; set; } public void Clear() { _selectedPokemon.Clear(); } protected override async Task OnInitializedAsync() { await GetPokemonListAsync(); } private Task\u0026lt;IEnumerable\u0026lt;PokemonInfo\u0026gt;?\u0026gt; Search(string value) { // if text is null or empty, show complete list if (string.IsNullOrEmpty(value)) return Task.FromResult(_pokemon?.Take(10)); return Task.FromResult(_pokemon? .Where(pokemon =\u0026gt; pokemon.Name.Contains(value, StringComparison.InvariantCultureIgnoreCase)) .Take(10)); } protected async Task GetPokemonListAsync() { _pokemon = await PokemonDataService.GetPokemonListAsync(); } } WhosThatPokemonResult WhosThatPokemonResult.razor Nothing fancy here, this component just shows the result after selecting a Pokémon.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 \u0026lt;div class=\u0026#34;container\u0026#34;\u0026gt; @if (Show) { @if (Correct) { \u0026lt;div class=\u0026#34;game-result-correct\u0026#34;\u0026gt; That\u0026#39;s right! \u0026lt;/div\u0026gt; } else { \u0026lt;div class=\u0026#34;game-result-wrong\u0026#34;\u0026gt; Sorry, it\u0026#39;s not \u0026lt;/div\u0026gt; } } \u0026lt;/div\u0026gt; WhosThatPokemonResult.razor.cs 1 2 3 4 5 6 7 8 9 10 11 using Microsoft.AspNetCore.Components; namespace BlazorHybridApp.Components; public partial class WhosThatPokemonResult { [Parameter] public bool Show { get; init; } [Parameter] public bool Correct { get; init; } } WhosThatPokemon WhosThatPokemon.razor This component uses all the other components, orchestrating their parameter values and events:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 \u0026lt;div class=\u0026#34;container\u0026#34;\u0026gt; @if (_loading) { \u0026lt;Loader\u0026gt;\u0026lt;/Loader\u0026gt; } else { \u0026lt;div class=\u0026#34;title\u0026#34;\u0026gt; Who\u0026#39;s that Pokémon?! \u0026lt;/div\u0026gt; \u0026lt;WhosThatPokemonData PokemonId=\u0026#34;_pokemonData.Id\u0026#34; PokemonName=\u0026#34;@_pokemonData.Name\u0026#34; Show=\u0026#34;_showResult\u0026#34;\u0026gt;\u0026lt;/WhosThatPokemonData\u0026gt; \u0026lt;PokemonSelector @ref=\u0026#34;_pokemonSelector\u0026#34;\u0026gt;\u0026lt;/PokemonSelector\u0026gt; \u0026lt;MudButton @onclick=\u0026#34;Confirm\u0026#34; class=\u0026#34;confirm-button\u0026#34; Variant=\u0026#34;Variant.Filled\u0026#34;\u0026gt;Confirm\u0026lt;/MudButton\u0026gt; \u0026lt;WhosThatPokemonResult Correct=\u0026#34;_correct\u0026#34; Show=\u0026#34;_showResult\u0026#34;\u0026gt;\u0026lt;/WhosThatPokemonResult\u0026gt; \u0026lt;MudButton class=\u0026#34;confirm-button\u0026#34; Variant=\u0026#34;Variant.Filled\u0026#34; @onclick=\u0026#34;ShowNextAsync\u0026#34;\u0026gt;Show Next\u0026lt;/MudButton\u0026gt; } \u0026lt;/div\u0026gt; WhosThatPokemon.razor.css Here we have to use ::deep to set the css properties for the MudButton. This is necessary because Blazor CSS isolation only works for elements directly below its component hierarchy in the DOM.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 .container { border-radius: 3px; background-color: rgba(255, 255, 255, 0.9); margin: 15px; padding: 10px; text-align: center; max-width: 280px; margin-left: auto; margin-right: auto; height: calc(100vh - 30px); } ::deep button.confirm-button { margin-top: 10px; background-color: red; color: white; margin: 5px; } .title { font-size: large; } WhosThatPokemon.razor.cs In the OnInitializedAsync and ShowNextAsync method, we select a random Pokémon that is passed to the other components.\nIn the Confirm method, we check if the Pokémon selected is the right one and pass the result to the WhosThatPokemonResult component.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 using BlazorHybridApp.Components.Domain; using BlazorHybridApp.Components.Services; using Microsoft.AspNetCore.Components; namespace BlazorHybridApp.Components; public partial class WhosThatPokemon { [Inject] private PokemonDataService PokemonDataService { get; init; } = default!; private PokemonInfo _pokemonData = default!; private PokemonSelector _pokemonSelector = default!; private bool _correct; private bool _showResult; private bool _loading; protected override async Task OnInitializedAsync() { _pokemonData = await GetRandomPokemon(); } private async Task\u0026lt;PokemonInfo\u0026gt; GetRandomPokemon() { try { _loading = true; var pokemonCount = await PokemonDataService.GetPokemonCountAsync(); Random r = new Random(); var pokemonId = r.Next(1, pokemonCount); return await PokemonDataService.GetPokemonByIdAsync(pokemonId); } finally { _loading = false; } } protected void Confirm() { _correct = _pokemonSelector?.SelectedPokemon?.Id == _pokemonData?.Id; _showResult = true; } protected async Task ShowNextAsync() { _showResult = false; _pokemonSelector.Clear(); _pokemonData = await GetRandomPokemon(); } } Other customizations Status bar color To change the status bar color for the app, we have to create a style tag and define the android:statusBarColor and android:navigationBarColor elements in the Platforms/Android/Resources/values/colors.xml file:\n1 2 3 4 5 6 7 8 9 10 11 \u0026lt;?xml version=\u0026#34;1.0\u0026#34; encoding=\u0026#34;utf-8\u0026#34;?\u0026gt; \u0026lt;resources\u0026gt; \u0026lt;color name=\u0026#34;colorPrimary\u0026#34;\u0026gt;#FF0000\u0026lt;/color\u0026gt; \u0026lt;color name=\u0026#34;colorPrimaryDark\u0026#34;\u0026gt;#DD0000\u0026lt;/color\u0026gt; \u0026lt;color name=\u0026#34;colorAccent\u0026#34;\u0026gt;#BB0000\u0026lt;/color\u0026gt; \u0026lt;style name=\u0026#34;MainTheme\u0026#34; parent=\u0026#34;@style/Maui.SplashTheme\u0026#34;\u0026gt; \u0026lt;item name=\u0026#34;android:statusBarColor\u0026#34;\u0026gt;#FF0000\u0026lt;/item\u0026gt; \u0026lt;item name=\u0026#34;android:navigationBarColor\u0026#34;\u0026gt;#FF0000\u0026lt;/item\u0026gt; \u0026lt;/style\u0026gt; \u0026lt;/resources\u0026gt; Then, set the Theme property of the Activity attribute to the style name (@style/MainTheme, in this example) in the MainActivity.cs file:\n1 2 3 4 [Activity(Theme = \u0026#34;@style/MainTheme\u0026#34;, MainLauncher = true, ConfigurationChanges = ConfigChanges.ScreenSize | ConfigChanges.Orientation | ConfigChanges.UiMode | ConfigChanges.ScreenLayout | ConfigChanges.SmallestScreenSize | ConfigChanges.Density)] public class MainActivity : MauiAppCompatActivity { } Running the app Since MAUI applications run on multiple platforms, we can run the app on Windows to debug or do quicker tests. Just select Windows Machine from the Run menu:\n💡 I recommend using an Android physical device when running/debugging on Android, as it is many times faster than running on an emulator. Just connect the device to the computer using an USB cable and the ADB drivers installed, and it will show up in the Run menu: Full source code GitHub repository\nReferences and Links What\u0026rsquo;s MAUI Blazor Hybrid MudBlazor docs Change a .NET MAUI app icon Add a splash screen to a .NET MAUI app project ","date":"2023-06-28T06:00:00-03:00","dateFormatted":"28 Jun 2023","image":"https://genezini.com/blog/p/building-a-whos-that-pokemon-game-for-android-with-net-maui-and-blazor-hybrid/cover.jpg","permalink":"https://genezini.com/blog/p/building-a-whos-that-pokemon-game-for-android-with-net-maui-and-blazor-hybrid/","title":"Building a Who's that Pokémon game for Android with .NET MAUI and Blazor Hybrid"},{"content":"Introduction Unlike MySQL and PostgreSQL, which execute scripts in the /docker-entrypoint-initdb.d/ directory after initialization, the SQL Server container image doesn\u0026rsquo;t provide a built-in mechanism for running scripts at startup.\nIn this post, I\u0026rsquo;ll show how to create a custom container image that waits for SQL Server startup and runs a pre-configured script. In addition, I\u0026rsquo;ll show how to use this custom image to run SQL Server in Testcontainers.\n⚠️ Starting with SQL Server 2022 CU 14 (and SQL Server 2019 CU 28), the container image moved sqlcmd from /opt/mssql-tools/bin to /opt/mssql-tools18/bin and enabled mandatory encryption by default. The code examples below have been updated to use the new path and the -C flag to trust the server certificate. If you\u0026rsquo;re using an older image (pre-CU 14), use /opt/mssql-tools/bin/sqlcmd without -C. See GitHub Issue #892 for details.\nHow to run scripts on SQL Server initialization The SQL Server Docker image documentation references this code as an example of how to run scripts on initialization, but the script is outdated and doesn\u0026rsquo;t work.\nI opened a Pull Request with a fixed version. It was never merged, but the fixes are incorporated into the approach below.\nBelow, I\u0026rsquo;ll explain each of the files.\n1 - Dockerfile The Dockerfile starts from the SQL Server 2022 image, copies all files into /tmp/initscripts, and sets the container entrypoint to entrypoint.sh.\nFROM mcr.microsoft.com/mssql/server:2022-latest # Bundle config source COPY . /tmp/initscripts ENTRYPOINT [\u0026#34;/tmp/initscripts/entrypoint.sh\u0026#34;] 2 - entrypoint.sh This bash script launches configure-db.sh in the background, then starts SQL Server in the foreground. Using exec ensures the SQL Server process replaces the shell as PID 1, so it receives Docker stop signals directly.\n#!/bin/bash # Start the script to create the DB and user /tmp/initscripts/configure-db.sh \u0026amp; # Start SQL Server (exec replaces shell with sqlservr as PID 1) exec /opt/mssql/bin/sqlservr 3 - configure-db.sh This is the most important part. The script waits up to 60 seconds (TRIES variable) for SQL Server to start and all databases to reach ONLINE state, then executes setup.sql. On success it logs Configuration completed.; on timeout it exits with code 1.\nℹ️ The Configuration completed message is important because we can use it to wait for the scripts to complete before accessing the database. I\u0026rsquo;ll use it in the next section when starting the container from Testcontainers. #!/bin/bash # Calls SQLCMD to verify that system and user databases return \u0026#34;0\u0026#34; which means all databases are in an \u0026#34;online\u0026#34; state, # then run the configuration script (setup.sql) # https://docs.microsoft.com/en-us/sql/relational-databases/system-catalog-views/sys-databases-transact-sql?view=sql-server-2017 TRIES=60 DBSTATUS=1 ERRCODE=1 i=0 while [[ $DBSTATUS -ne 0 ]] \u0026amp;\u0026amp; [[ $i -lt $TRIES ]]; do i=$((i+1)) DBSTATUS=$(/opt/mssql-tools18/bin/sqlcmd -h -1 -t 2 -C -S localhost -U sa -P \u0026#34;$MSSQL_SA_PASSWORD\u0026#34; -Q \u0026#34;SET NOCOUNT ON; Select COALESCE(SUM(state), 0) from sys.databases\u0026#34;) || DBSTATUS=1 sleep 1s done if [ $DBSTATUS -ne 0 ]; then echo \u0026#34;SQL Server took more than $TRIES seconds to start up or one or more databases are not in an ONLINE state\u0026#34; exit 1 fi # Run the setup script to create the DB and the schema in the DB echo \u0026#34;Running configuration script...\u0026#34; /opt/mssql-tools18/bin/sqlcmd -C -S localhost -U sa -P \u0026#34;$MSSQL_SA_PASSWORD\u0026#34; -d master -i setup.sql echo \u0026#34;Configuration completed.\u0026#34; 4 - setup.sql This is the script that will be run. Here I\u0026rsquo;m just creating the database.\nCREATE DATABASE [MyDatabase] GO 5 - Building the container image Now, we have to build the container image from the Dockerfile we created. In this example, I\u0026rsquo;m naming the image mydatabase-sqlserver:\ndocker build -t mydatabase-sqlserver . 6 - Starting the container Finally, we can run the container with docker run:\ndocker run -e \u0026#34;ACCEPT_EULA=Y\u0026#34; -e \u0026#34;MSSQL_SA_PASSWORD=yourStrong(!)Password\u0026#34; -p 1433:1433 mydatabase-sqlserver Or create a compose.yaml file:\nservices: sqlserver: build: . environment: MSSQL_SA_PASSWORD: \u0026#34;yourStrong(!)Password\u0026#34; ACCEPT_EULA: \u0026#34;Y\u0026#34; MSSQL_PID: \u0026#34;Developer\u0026#34; ports: - \u0026#34;1433:1433\u0026#34; And run:\ndocker compose up Source code of this example https://github.com/dgenezini/SqlServerScriptOnStartup Running the configuration script in Testcontainers Testcontainers is a library that provides lightweight, throwaway instances of databases, selenium web browsers, or anything that can run in a container. These instances can be especially useful for testing applications against real dependencies, like databases, that can be created and disposed of after the tests.\nI explained about Testcontainers in this post.\nI strongly recommend reading it first. Here, I\u0026rsquo;ll cover only the specifics of running SQL Server with configuration scripts in Testcontainers.\nStarting the container When starting the container, we need to override two configurations in MsSqlBuilder:\nWithImage overrides MsSqlBuilder\u0026rsquo;s default image with our custom image (mydatabase-sqlserver) WithWaitStrategy overrides the default wait strategy with UntilMessageIsLogged. This makes StartAsync block until Configuration completed appears in the container logs, ensuring the setup script finishes before we use the container. 1 2 3 4 5 6 7 var MsSqlContainer = new MsSqlBuilder() .WithImage(\u0026#34;mydatabase-sqlserver\u0026#34;) .WithPassword(\u0026#34;SqlServer2022!Password\u0026#34;) .WithWaitStrategy(Wait.ForUnixContainer().UntilMessageIsLogged(\u0026#34;Configuration completed\u0026#34;)) .Build(); await MsSqlContainer.StartAsync(); ℹ️ The DockerContainer class also provides ExecScriptAsync to run scripts inside a running container, but the goal here is to have a script that runs automatically—both from Testcontainers and when running the container directly (e.g., during local development). Source code of this example https://github.com/dgenezini/TestcontainersSqlServerSample ","date":"2023-05-15T06:00:00-03:00","dateFormatted":"15 May 2023","image":"https://genezini.com/blog/p/how-to-run-scripts-on-sql-server-container-startup/cover.jpg","permalink":"https://genezini.com/blog/p/how-to-run-scripts-on-sql-server-container-startup/","title":"How to run scripts on SQL Server container startup"},{"content":"On a branch drifted behind a busy main, git rebase origin/main conflicts on the first commit and again on each remaining one — 16 instead of one. Rebase an empty branch and replay the work as one uncommitted change.\ngit checkout branch-to-rebase git fetch --all git merge-base branch-to-rebase origin/main # last commit before the branch\u0026#39;s own work Reset to it: the branch\u0026rsquo;s differences return to the working tree, unstaged:\ngit reset f9fb326cb6cd58e0f31b433389b4a76f60319db1 Stash it; the rebase has nothing to conflict over:\ngit stash git rebase origin/main # Successfully rebased and updated refs/heads/branch-to-rebase git stash pop # conflicts now appear exactly once Resolve, commit, push:\ngit commit git push --force Two side effects: a squash rebase (one commit), and the original branch is gone — use git checkout -b clean-branch origin/main instead of the rebase when it matters.\nThe two aliases mb returns the merge-base of the current branch, reset-to-base feeds it to reset:\nmb = \u0026#34;!f() { git branch --show-current | xargs git merge-base $1; }; f\u0026#34; reset-to-base = \u0026#34;!f() { git mb $1 | xargs git reset; }; f\u0026#34; git mb origin/main # f2732648ef5b6804a63d44f1b498f19ddf01eeb6 git reset-to-base origin/main # reset to that base Both are in the aliases note with the rest.\nReferences How to Skip Rebase Conflicts — AJ ONeal ","date":"2023-04-18T00:00:00Z","dateFormatted":"18 Apr 2023","permalink":"https://genezini.com/garden/git-rebase/","title":"Rebasing a messy branch in one conflict pass"},{"content":"Introduction WireMock.Net is a great tool to remove external dependencies when writing integration tests, but because it is highly configurable, it can be hard to find why its mocks aren\u0026rsquo;t working.\nIn this post, I\u0026rsquo;ll explain how to troubleshoot problems in its configuration and show some common problems that happen in my day-to-day work.\nWhat is WireMock.Net? WireMock.Net is a library for stubbing and mocking HTTP APIs. I wrote about how and why to use it previously, and will use the previous post examples in this one.\nWireMock.Net\u0026rsquo;s Admin API WireMock.Net has an Admin API that is essential for debugging problems in our mocks.\nThe API offers many endpoints, but I\u0026rsquo;ll show the two that will be used to find problems in the mocks.\nℹ️ The complete list of endpoints can be seen in the WireMock.Net\u0026rsquo;s documentation. Mappings endpoint The endpoint /__admin/mappings returns the mappings configured for the mocks, including all the matchers that need to be fulfilled for the mock to respond, and the response that will be returned.\nHere is an example of the return:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 [ { \u0026#34;Guid\u0026#34;: \u0026#34;4dbbabf7-aac9-4f31-9fb9-a02360a454ea\u0026#34;, \u0026#34;Request\u0026#34;: { \u0026#34;Path\u0026#34;: { \u0026#34;Matchers\u0026#34;: [ { \u0026#34;Name\u0026#34;: \u0026#34;WildcardMatcher\u0026#34;, \u0026#34;Pattern\u0026#34;: \u0026#34;/pokemon/charmander\u0026#34;, \u0026#34;IgnoreCase\u0026#34;: false } ] }, \u0026#34;Methods\u0026#34;: [ \u0026#34;GET\u0026#34; ] }, \u0026#34;Response\u0026#34;: { \u0026#34;StatusCode\u0026#34;: 200, \u0026#34;BodyDestination\u0026#34;: \u0026#34;SameAsSource\u0026#34;, \u0026#34;Body\u0026#34;: \u0026#34;{\\\u0026#34;abilities\\\u0026#34;:[{\\\u0026#34;ability\\\u0026#34;:{\\\u0026#34;name\\\u0026#34;:\\\u0026#34;name8f22045d-c183-4f1c-a32d-aaa6d337a9b7\\\u0026#34;,\\\u0026#34;url\\\u0026#34;:\\\u0026#34;url768c48e4-0dec-4952-b810-06472f602b4d\\\u0026#34;},\\\u0026#34;is_hidden\\\u0026#34;:true,\\\u0026#34;slot\\\u0026#34;:192}, ... \\\u0026#34;url\\\u0026#34;:\\\u0026#34;url716f6971-abd8-4e35-a9af-b7bd7a9a9cd6\\\u0026#34;}}],\\\u0026#34;weight\\\u0026#34;:128}\u0026#34;, \u0026#34;Headers\u0026#34;: { \u0026#34;Content-Type\u0026#34;: \u0026#34;application/json\u0026#34; } }, \u0026#34;UseWebhooksFireAndForget\u0026#34;: false } ] Requests endpoint The endpoint /__admin/requests returns the history of requests made to WireMock. It includes information about the request made and the response received by the requester.\nIt also includes the PartialRequestMatchResult property, that shows which matcher was successful and which was not.\nHere is an example of a request that reached the mock:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 [ { \u0026#34;Guid\u0026#34;: \u0026#34;d2759bea-e4cc-442b-a63c-506ac6d61527\u0026#34;, \u0026#34;Request\u0026#34;: { \u0026#34;ClientIP\u0026#34;: \u0026#34;::1\u0026#34;, \u0026#34;DateTime\u0026#34;: \u0026#34;2023-04-09T21:23:52.2879698Z\u0026#34;, \u0026#34;Path\u0026#34;: \u0026#34;/pokemon/charmander\u0026#34;, \u0026#34;AbsolutePath\u0026#34;: \u0026#34;/pokemon/charmander\u0026#34;, \u0026#34;Url\u0026#34;: \u0026#34;http://localhost:64378/pokemon/charmander\u0026#34;, \u0026#34;AbsoluteUrl\u0026#34;: \u0026#34;http://localhost:64378/pokemon/charmander\u0026#34;, \u0026#34;Query\u0026#34;: {}, \u0026#34;Method\u0026#34;: \u0026#34;GET\u0026#34;, \u0026#34;Headers\u0026#34;: { \u0026#34;Host\u0026#34;: [ \u0026#34;localhost:64378\u0026#34; ], \u0026#34;traceparent\u0026#34;: [ \u0026#34;00-fc2bc88c5bfc8b273e12d5a64cfd67cc-4ffca316becbeb2b-00\u0026#34; ] }, \u0026#34;Cookies\u0026#34;: {} }, \u0026#34;Response\u0026#34;: { \u0026#34;StatusCode\u0026#34;: 200, \u0026#34;Headers\u0026#34;: { \u0026#34;Content-Type\u0026#34;: [ \u0026#34;application/json\u0026#34; ] }, \u0026#34;BodyDestination\u0026#34;: \u0026#34;SameAsSource\u0026#34;, \u0026#34;Body\u0026#34;: \u0026#34;{\\\u0026#34;abilities\\\u0026#34;:[{\\\u0026#34;ability\\\u0026#34;:{\\\u0026#34;name\\\u0026#34;:\\\u0026#34;name8f22045d-c183-4f1c-a32d-aaa6d337a9b7\\\u0026#34;,\\\u0026#34;url\\\u0026#34;:\\\u0026#34;url768c48e4-0dec-4952-b810-06472f602b4d\\\u0026#34;},\\\u0026#34;is_hidden\\\u0026#34;:true,\\\u0026#34;slot\\\u0026#34;:192},{\\\u0026#34;ability\\\u0026#34;:{\\\u0026#34;name\\\u0026#34;:\\\u0026#34;name7e833a25-8050-4369-a5b9-811b4b00ee69\\\u0026#34;, ... \\\u0026#34;url\\\u0026#34;:\\\u0026#34;url716f6971-abd8-4e35-a9af-b7bd7a9a9cd6\\\u0026#34;}}],\\\u0026#34;weight\\\u0026#34;:128}\u0026#34;, \u0026#34;BodyEncoding\u0026#34;: { \u0026#34;CodePage\u0026#34;: 65001, \u0026#34;EncodingName\u0026#34;: \u0026#34;Unicode (UTF-8)\u0026#34;, \u0026#34;WebName\u0026#34;: \u0026#34;utf-8\u0026#34; }, \u0026#34;DetectedBodyType\u0026#34;: 1 }, \u0026#34;MappingGuid\u0026#34;: \u0026#34;4dbbabf7-aac9-4f31-9fb9-a02360a454ea\u0026#34;, \u0026#34;RequestMatchResult\u0026#34;: { \u0026#34;TotalScore\u0026#34;: 2.0, \u0026#34;TotalNumber\u0026#34;: 2, \u0026#34;IsPerfectMatch\u0026#34;: true, \u0026#34;AverageTotalScore\u0026#34;: 1.0, \u0026#34;MatchDetails\u0026#34;: [ { \u0026#34;Name\u0026#34;: \u0026#34;PathMatcher\u0026#34;, \u0026#34;Score\u0026#34;: 1.0 }, { \u0026#34;Name\u0026#34;: \u0026#34;MethodMatcher\u0026#34;, \u0026#34;Score\u0026#34;: 1.0 } ] }, \u0026#34;PartialMappingGuid\u0026#34;: \u0026#34;4dbbabf7-aac9-4f31-9fb9-a02360a454ea\u0026#34;, \u0026#34;PartialRequestMatchResult\u0026#34;: { \u0026#34;TotalScore\u0026#34;: 2.0, \u0026#34;TotalNumber\u0026#34;: 2, \u0026#34;IsPerfectMatch\u0026#34;: true, \u0026#34;AverageTotalScore\u0026#34;: 1.0, \u0026#34;MatchDetails\u0026#34;: [ { \u0026#34;Name\u0026#34;: \u0026#34;PathMatcher\u0026#34;, \u0026#34;Score\u0026#34;: 1.0 }, { \u0026#34;Name\u0026#34;: \u0026#34;MethodMatcher\u0026#34;, \u0026#34;Score\u0026#34;: 1.0 } ] } } ] The RequestMatchResult shows that the mock has 2 matchers configured (TotalNumber) and the total score was 2 (TotalScore). The MatchDetails also shows a score of 1.0 (100%) for all the matchers.\nHere is an example of a request that didn\u0026rsquo;t fulfill all of the matchers:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 [ { \u0026#34;Guid\u0026#34;: \u0026#34;60812be4-2d28-436b-afe3-4597b57e1995\u0026#34;, \u0026#34;Request\u0026#34;: { \u0026#34;ClientIP\u0026#34;: \u0026#34;::1\u0026#34;, \u0026#34;DateTime\u0026#34;: \u0026#34;2023-04-10T22:09:24.4115811Z\u0026#34;, \u0026#34;Path\u0026#34;: \u0026#34;/pokemon/squirtle\u0026#34;, \u0026#34;AbsolutePath\u0026#34;: \u0026#34;/pokemon/squirtle\u0026#34;, \u0026#34;Url\u0026#34;: \u0026#34;http://localhost:57546/pokemon/squirtle\u0026#34;, \u0026#34;AbsoluteUrl\u0026#34;: \u0026#34;http://localhost:57546/pokemon/squirtle\u0026#34;, \u0026#34;Query\u0026#34;: {}, \u0026#34;Method\u0026#34;: \u0026#34;GET\u0026#34;, \u0026#34;Headers\u0026#34;: { \u0026#34;Host\u0026#34;: [ \u0026#34;localhost:57546\u0026#34; ], \u0026#34;traceparent\u0026#34;: [ \u0026#34;00-f89a0136b596e297767aea9602ed6df2-3e4ef33765b915d1-00\u0026#34; ] }, \u0026#34;Cookies\u0026#34;: {} }, \u0026#34;Response\u0026#34;: { \u0026#34;StatusCode\u0026#34;: 404, \u0026#34;Headers\u0026#34;: { \u0026#34;Content-Type\u0026#34;: [ \u0026#34;application/json\u0026#34; ] }, \u0026#34;BodyAsJson\u0026#34;: { \u0026#34;Status\u0026#34;: \u0026#34;No matching mapping found\u0026#34; }, \u0026#34;DetectedBodyType\u0026#34;: 2 }, \u0026#34;PartialMappingGuid\u0026#34;: \u0026#34;bab89116-6318-4ecb-8460-39e5116aeaec\u0026#34;, \u0026#34;PartialRequestMatchResult\u0026#34;: { \u0026#34;TotalScore\u0026#34;: 1.0, \u0026#34;TotalNumber\u0026#34;: 2, \u0026#34;IsPerfectMatch\u0026#34;: false, \u0026#34;AverageTotalScore\u0026#34;: 0.5, \u0026#34;MatchDetails\u0026#34;: [ { \u0026#34;Name\u0026#34;: \u0026#34;PathMatcher\u0026#34;, \u0026#34;Score\u0026#34;: 0.0 }, { \u0026#34;Name\u0026#34;: \u0026#34;MethodMatcher\u0026#34;, \u0026#34;Score\u0026#34;: 1.0 } ] } } ] The PartialRequestMatchResult shows that the mock has 2 matchers configured (TotalNumber) and the total score was 1 (TotalScore). In the MatchDetails we can see that the problem was on the PathMatcher.\nLooking in the mappings endpoint, we see that the path was configured to /pokemon/charmander, instead of the /pokemon/squirtle that was in the request.\nConfiguring the Admin Interface To use the admin interface, we just need to start WireMock\u0026rsquo;s server with the StartWithAdminInterface method instead of the Start method:\nvar wiremockServer = WireMockServer.StartWithAdminInterface(); Then, to be able to access the admin interface, we include a delay after the Act part of the test:\nawait Task.Delay(TimeSpan.FromMinutes(100)); Lastly, we access the endpoint using the URL returned by the Url property of the wireMockSvr object:\nFull test code:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 [Fact] public async Task Get_Existing_Pokemon_Returns_200() { //Arrange var wireMockSvr = WireMockServer.StartWithAdminInterface(); //Start WireMock with Admin Interface var Factory = _factory .WithWebHostBuilder(builder =\u0026gt; { builder.UseSetting(\u0026#34;PokeApiBaseUrl\u0026#34;, wireMockSvr.Url); }); var HttpClient = Factory.CreateClient(); Fixture fixture = new Fixture(); var ResponseObj = fixture.Create\u0026lt;PokemonInfo\u0026gt;(); var ResponseObjJson = JsonSerializer.Serialize(ResponseObj); wireMockSvr .Given(Request.Create() .WithPath(\u0026#34;/pokemon/charmander\u0026#34;) .UsingGet()) .RespondWith(Response.Create() .WithBody(ResponseObjJson) .WithHeader(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;) .WithStatusCode(HttpStatusCode.OK)); //Act var HttpResponse = await HttpClient.GetAsync(\u0026#34;/pokemoninfo/charmander\u0026#34;); await Task.Delay(TimeSpan.FromMinutes(30)); //Delay to be able to examine the admin interface //Assert HttpResponse.StatusCode.Should().Be(HttpStatusCode.OK); var ResponseJson = await HttpResponse.Content.ReadAsStringAsync(); var PokemonInfo = JsonSerializer.Deserialize\u0026lt;PokemonInfo\u0026gt;(ResponseJson); PokemonInfo.Should().BeEquivalentTo(ResponseObj); wireMockSvr.Stop(); } Common problems Query string params Let\u0026rsquo;s take the endpoint /pokemon?type={typeName} as an example.\nThe mock below won\u0026rsquo;t work:\n1 2 3 4 5 6 7 8 WireMockSvr .Given(Request.Create() .WithPath(\u0026#34;/pokemon?type=fire\u0026#34;) .UsingGet()) .RespondWith(Response.Create() .WithBody(ResponseObjJson) .WithHeader(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;) .WithStatusCode(HttpStatusCode.OK)); To create a mock for endpoints with query strings, we have to use the WithParam property:\n1 2 3 4 5 6 7 8 9 WireMockSvr .Given(Request.Create() .WithPath(\u0026#34;/pokemon\u0026#34;) .WithParam(\u0026#34;type\u0026#34;, \u0026#34;fire\u0026#34;) .UsingGet()) .RespondWith(Response.Create() .WithBody(ResponseObjJson) .WithHeader(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;) .WithStatusCode(HttpStatusCode.OK)); Params in the route Contrary to params in the query string, params in the path won\u0026rsquo;t work when configured with WithParam. For example, the endpoint /pokemon/{pokemonName} won\u0026rsquo;t be reached with:\n1 2 3 4 5 6 7 8 9 WireMockSvr .Given(Request.Create() .WithPath(\u0026#34;/pokemon\u0026#34;) .WithParam(\u0026#34;pokemonName\u0026#34;, \u0026#34;charmander\u0026#34;) .UsingGet()) .RespondWith(Response.Create() .WithBody(ResponseObjJson) .WithHeader(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;) .WithStatusCode(HttpStatusCode.OK)); It has to be set in the WithPath method:\n1 2 3 4 5 6 7 8 WireMockSvr .Given(Request.Create() .WithPath(\u0026#34;/pokemon/charmander\u0026#34;) .UsingGet()) .RespondWith(Response.Create() .WithBody(ResponseObjJson) .WithHeader(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;) .WithStatusCode(HttpStatusCode.OK)); Running behind a network proxy When running behind a network proxy, WireMock may be unreachable, causing a timeout in the application.\nTo ignore the proxy for localhost, we need to configure the no_proxy environment variable, adding localhost to its value (more values can be included, separated by comma):\nShared WireMock server for all tests Sharing WireMock\u0026rsquo;s server instance between tests can cause random problems because mock definitions are overridden when configured for the second time. For example, take two tests running in parallel:\nTest 1 configures /pokemon/charmander to return status 200; Test 2 configures /pokemon/charmander to return status 404. The first test to configure the mock will break because its mock will be overridden and the result won\u0026rsquo;t be as expected.\nTo avoid this random problems, we need to use one WireMock instance for each test:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 [Fact] public async Task Get_Existing_Pokemon_Returns_200() { var WireMockSvr = WireMockServer.StartWithAdminInterface(); ... WireMockSvr.Stop(); } [Fact] public async Task Get_NotExisting_Pokemon_Returns_404() { var WireMockSvr = WireMockServer.StartWithAdminInterface(); ... WireMockSvr.Stop(); } References and Links WireMock\u0026rsquo;s Admin API Documentation ","date":"2023-04-11T08:00:00-03:00","dateFormatted":"11 Apr 2023","image":"https://genezini.com/blog/p/why-my-wiremock-mocks-arent-working/cover.jpg","permalink":"https://genezini.com/blog/p/why-my-wiremock-mocks-arent-working/","title":"Why my WireMock mocks aren't working?"},{"content":"Introduction LocalStack is an open-source framework that allows us to emulate the major AWS services locally, making it easier to develop and test cloud applications without incurring the cost and complexity of deploying to a real cloud environment.\nIn this post, I\u0026rsquo;ll show how to configure it to emulate S3 buckets and how to interact with those buckets from a C# application.\nRunning LocalStack LocalStack can be run as a CLI or using a container. In this post, I\u0026rsquo;ll explain how to run it in a container with docker run and docker-compose.\nℹ️ If you don\u0026rsquo;t have docker or other container runtime installed, click here for Docker installation instructions or here for Podman installation instructions. Container To start a container with a LocalStack instance, run the following command:\ndocker run --rm -it -p 4566:4566 -p 4510-4559:4510-4559 -e EXTRA_CORS_ALLOWED_ORIGINS=https://app.localstack.cloud. localstack/localstack:1.3.1 Note that it is exposing port 4566 and ports 4510 to 4559 and allowing CORS access from https://app.localstack.cloud. (to allow access from the LocalStack dashboard).\n🚨 It\u0026rsquo;s recommended to use a specific version instead of latest to avoid problems with new versions updated automatically. Docker compose Starting LocalStack from docker compose is just as easy. Just add the localstack service, as below, to a docker-compose.yaml file:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 version: \u0026#34;3.8\u0026#34; services: localstack: container_name: \u0026#34;${LOCALSTACK_DOCKER_NAME-localstack_main}\u0026#34; image: localstack/localstack ports: - \u0026#34;127.0.0.1:4566:4566\u0026#34; # LocalStack Gateway - \u0026#34;127.0.0.1:4510-4559:4510-4559\u0026#34; # external services port range environment: - DEBUG=${DEBUG-} - LAMBDA_EXECUTOR=${LAMBDA_EXECUTOR-} - DOCKER_HOST=unix:///var/run/docker.sock - EXTRA_CORS_ALLOWED_ORIGINS=https://app.localstack.cloud. # Enable access from the dashboard volumes: - \u0026#34;${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack\u0026#34; - \u0026#34;/var/run/docker.sock:/var/run/docker.sock\u0026#34; Then run the following command:\ndocker-compose up 🚨 It\u0026rsquo;s recommended to use a specific version instead of latest to avoid problems with new versions updated automatically. ⚠️ I\u0026rsquo;ve added the environment variable EXTRA_CORS_ALLOWED_ORIGINS with the value https://app.localstack.cloud. to allow access from the LocalStack dashboard. 💡 The updated docker-compose.yaml file can be found here, in the LocalStack repo. LocalStack Dashboard LocalStack has a web-based dashboard that allows us to manage and configure its services and visualize its logs.\nAccess https://app.localstack.cloud. and it will connect to the LocalStack instance running locally.\nIt runs in the browser, so there is no need to expose any ports to the internet.\n🚨 If the dashboard says \u0026ldquo;Please start LocalStack to check the System Status\u0026rdquo; and the container log shows Blocked CORS request from forbidden origin https://app.localstack.cloud., the EXTRA_CORS_ALLOWED_ORIGINS environment variable was not correctly set to https://app.localstack.cloud.. See here. Interacting with LocalStack using the AWS CLI We\u0026rsquo;ll use the AWS CLI to interact with LocalStack. If you don\u0026rsquo;t have the AWS CLI, look here for instructions on how to install it.\nConfiguring a profile for LocalStack The AWS CLI requires credentials when running. LocalStack doesn\u0026rsquo;t validate the credentials by default, so will create a profile with anything as access key and secret key just to make the CLI happy.\nIn a terminal, type aws configure --profile localstack; For AWS Access Key ID [None]:, type anything; For AWS Secret Access Key [None]:, type anything; For Default region name [None]:, type the region you prefer (for example, us-east-1); For Default output format [None]:, type json. How to create a bucket using the AWS CLI To create a S3 bucket in LocalStack, we\u0026rsquo;ll use the aws s3 mb command (mb is short for Make Bucket).\nThe command below will create a bucket with the name local-bucket-name using the AWS CLI profile we previously created with the name localstack. It\u0026rsquo;s important to pass the --endpoint parameter or else it will try to create the bucket in AWS.\naws s3 mb s3://local-bucket-name --endpoint http://localhost:4566 --profile localstack Looking at LocalStack dashboard we can see the bucket was created:\nHow to list a bucket contents using the AWS CLI To look the contents of a bucket, we can use the aws s3 ls command:\naws s3 ls s3://local-bucket-name --endpoint http://localhost:4566 --profile localstack Or use the LocalStack dashboard:\nAccessing LocalStack from .NET To access an S3 bucket from LocalStack in .NET we use the same libraries we use to access it in AWS.\nIn this example, I\u0026rsquo;ll use the AWSSDK.S3 and the AWSSDK.Extensions.NETCore.Setup NuGet packages, both from AWS.\nHow does the AmazonS3Client get AWS access data? When running in AWS, the AmazonS3Client will get its access data from the IAM Role attached to the service running it. When running locally, it will get from the AWS CLI profile named default or from the settings we pass to it.\nIn the code below, I\u0026rsquo;m checking for a configuration section with the name AWS, that is not present in the production environment. If it\u0026rsquo;s found, I set the Region, ServiceURL and ForcePathStyle properties of the AmazonS3Config and pass it to the creation of the AmazonS3Client.\nProgram.cs 1 2 3 4 5 var builder = WebApplication.CreateBuilder(args); builder.AddAwsServices(); var app = builder.Build(); AwsExtensions.cs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 public static class AwsExtensions { public static void AddAwsServices(this WebApplicationBuilder builder) { builder.Services.AddSingleton\u0026lt;IAmazonS3\u0026gt;(sc =\u0026gt; { var configuration = sc.GetRequiredService\u0026lt;IConfiguration\u0026gt;(); var awsConfiguration = configuration.GetSection(\u0026#34;AWS\u0026#34;).Get\u0026lt;AwsConfiguration\u0026gt;(); if (awsConfiguration?.ServiceURL is null) { return new AmazonS3Client(); } else { return AwsS3ClientFactory.CreateAwsS3Client( awsConfiguration.ServiceURL, awsConfiguration.Region, awsConfiguration.ForcePathStyle, awsConfiguration.AwsAccessKey, awsConfiguration.AwsSecretKey); } }); } } The appsettings.Development.json has the configurations pointing to the LocalStack instance:\nappsettings.Development.json 1 2 3 4 5 6 7 8 9 10 11 { \u0026#34;BucketName\u0026#34;: \u0026#34;local-bucket-name\u0026#34;, \u0026#34;AWS\u0026#34;: { \u0026#34;Region\u0026#34;: \u0026#34;us-east-1\u0026#34;, \u0026#34;ServiceURL\u0026#34;: \u0026#34;http://localhost:4566\u0026#34;, \u0026#34;ForcePathStyle\u0026#34;: \u0026#34;true\u0026#34;, \u0026#34;AwsAccessKey\u0026#34;: \u0026#34;test\u0026#34;, \u0026#34;AwsSecretKey\u0026#34;: \u0026#34;test\u0026#34; } } ⚠️ The ForcePathStyle forces the use of https://s3.amazonaws.com/\u0026lt;bucket-name\u0026gt;/\u0026lt;object-key\u0026gt; styled URLs instead of https://\u0026lt;bucket-name\u0026gt;.s3.amazonaws.com/\u0026lt;object-key\u0026gt; URLs. The ForcePathStyle needs to be set to true for the AmazonS3Client to work with LocalStack.\nUpload an image to the S3 Bucket Using Minimal APIs, I created an endpoint that receives a file and saves it in the S3 bucket.\nThe code is straight forward:\nProgram.cs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 app.MapPost(\u0026#34;/upload\u0026#34;, async (IAmazonS3 s3Client, IFormFile file) =\u0026gt; { var bucketName = builder.Configuration[\u0026#34;BucketName\u0026#34;]!; var bucketExists = await s3Client.DoesS3BucketExistAsync(bucketName); if (!bucketExists) { return Results.BadRequest($\u0026#34;Bucket {bucketName} does not exists.\u0026#34;); } using var fileStream = file.OpenReadStream(); var putObjectRequest = new PutObjectRequest() { BucketName = bucketName, Key = file.FileName, InputStream = fileStream }; putObjectRequest.Metadata.Add(\u0026#34;Content-Type\u0026#34;, file.ContentType); var putResult = await s3Client.PutObjectAsync(putObjectRequest); return Results.Ok($\u0026#34;File {file.FileName} uploaded to S3 successfully!\u0026#34;); }); Now, we can test it from Postman:\nGet an image from the S3 Bucket I also created an endpoint that returns the file with the key passed by parameter from the S3 bucket:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 app.MapGet(\u0026#34;/object/{key}\u0026#34;, async (IAmazonS3 s3Client, string key) =\u0026gt; { var bucketName = builder.Configuration[\u0026#34;BucketName\u0026#34;]!; var bucketExists = await s3Client.DoesS3BucketExistAsync(bucketName); if (!bucketExists) { return Results.BadRequest($\u0026#34;Bucket {bucketName} does not exists.\u0026#34;); } try { var getObjectResponse = await s3Client.GetObjectAsync(bucketName, key); return Results.File(getObjectResponse.ResponseStream, getObjectResponse.Headers.ContentType); } catch (AmazonS3Exception ex) when (ex.ErrorCode.Equals(\u0026#34;NotFound\u0026#34;, StringComparison.OrdinalIgnoreCase)) { return Results.NotFound(); } }); Testing from Postman, we can see the image previously uploaded:\nFull source code GitHub repository\nReferences and Links LocalStack Website LocalStack Feature Coverage LocalStack Dashboard awslocal CLI ","date":"2023-02-13T07:30:00-03:00","dateFormatted":"13 Feb 2023","image":"https://genezini.com/blog/p/dotnet-and-aws-s3-with-localstack-how-to-develop-with-local-s3-buckets/cover.jpg","permalink":"https://genezini.com/blog/p/dotnet-and-aws-s3-with-localstack-how-to-develop-with-local-s3-buckets/","title":".NET and AWS S3 with LocalStack: How to develop with local S3 buckets"},{"content":"Introduction At each release, C# adds features that help us make our codes cleaner, more readable and more maintainable. The problem is that, because some features are dependent of runtime implementations, C# versions are generally tied to .NET runtime versions. For example, C# 11 is enabled only in .NET 7 and above.\nIn this post, I\u0026rsquo;ll show how to use C# 11 in older runtime version (even .NET Framework 2.0).\nWhy not upgrade the .NET version? Upgrading to the newest .NET version is the best option. Not only we benefit from new C# features, but also from performance and security improvements.\nBut there are some scenarios where upgrading is not an option because of compatibility or because the cost of upgrading would be too high.\nSome examples are:\nAWS Lambda Functions running on .NET 6 (AWS Lambda doesn\u0026rsquo;t support .NET 7 at the time of this post); Plugins or extensions for proprietary software, such as Dynamics/Dataverse plugins that are not compatible with .NET Core Legacy systems with a large codebase that still receive frequent updates. What C# 11 features can be used? C# features are divided in features that require runtime support and features that are just syntactic sugar.\nFeatures that require runtime support cannot be used in older .NET versions, but most of the features that are syntactic sugar are compiled to IL (.NET Intermediate Language) and interpreted by older .NET versions at runtime (Even .NET Framework 2.0), depending only on an updated version of Roslyn (the .NET compiler) to work.\nHow to use C# 11 features in .NET 6 and previous versions Some features will work just by having the .NET 7 SDK installed and adding (or updating) the LangVersion tag to 11 in the csproj file.\n\u0026lt;LangVersion\u0026gt;11\u0026lt;/LangVersion\u0026gt; Examples Here are some examples of the most useful features of latest versions of C# (not only C# 11).\nTop-level statements No need to static void Main:\n1 2 3 4 5 using System; Console.WriteLine(\u0026#34;Hello World\u0026#34;); Console.ReadKey(); Nullable reference types This is a working example of nullable reference types and top-level statements in .NET Framework 2.0:\n1 2 3 4 5 6 7 8 9 #nullable enable using System; string? nullHere = null; Console.WriteLine($\u0026#34;Length: {nullHere?.Length}\u0026#34;); Console.ReadKey(); 💡 We can even treat warnings as errors, as I explained in this post. ℹ️ If the project doesn\u0026rsquo;t use the new csproj format, the \u0026lt;Nullable\u0026gt;enable\u0026lt;/Nullable\u0026gt; won\u0026rsquo;t be interpreted and the use of the #nullable enable directive at the start of each file is required. ⚠️ One caveat of using nullable types in older .NET versions is that framework functions won\u0026rsquo;t inform us if they return nullable reference types because these changes were implemented only in the newer .NET versions. 💡 EDIT: A reader reached me about this cool package that partially solves this problem by injecting nullable reference type annotations in CLR\u0026rsquo;s methods of some assemblies (check the docs for more details): ReferenceAssemblyAnnotator. Pattern Matching Pattern matching also works in .NET Framework 2.0:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 using System; Console.WriteLine($\u0026#34;Enter the water temperature in Fahrenheit:\u0026#34;); var isNumber = int.TryParse(Console.ReadLine(), out var number); string GetWaterState(int tempInFahrenheit) =\u0026gt; tempInFahrenheit switch { (\u0026gt; 32) and (\u0026lt; 212) =\u0026gt; \u0026#34;liquid\u0026#34;, \u0026lt; 32 =\u0026gt; \u0026#34;solid\u0026#34;, \u0026gt; 212 =\u0026gt; \u0026#34;gas\u0026#34;, 32 =\u0026gt; \u0026#34;solid/liquid transition\u0026#34;, 212 =\u0026gt; \u0026#34;liquid / gas transition\u0026#34; }; if (isNumber) { var waterState = GetWaterState(number); Console.WriteLine(waterState); } else { Console.WriteLine(\u0026#34;Invalid number\u0026#34;); } Console.ReadKey(); ⚠️ List pattern matching won\u0026rsquo;t work just by changing the LangVersion tag. It needs specific types that I\u0026rsquo;ll explain in the next section. Features that need specific types Even for features that are syntactic sugar, some depend on types and attributes implemented in the newer CLRs (for instance, the list pattern matching and the required keyword).\nIf we copy those types from the CLR source code or reference them from NuGet packages, the compilation will succeed and the features will be available.\nBut there is a better alternative\u0026hellip;\nEnter PolySharp PolySharp is a NuGet package created by Sergio Pedri, Software Engineer at Microsoft, that generates polyfills for those types at compile time, only for the features being used in the code and that are not present in the targeted runtime.\nFeatures enabled by PolySharp This is a shortened list of some C# features enabled by PolySharp:\nNullability attributes Index and Range List pattern matching Required members Init-only properties [CallerArgumentExpression] [StringSyntax] To install it, just add its NuGet package:\nInstall-Package PolySharp ⚠️ Because PolySharp uses source generators, it doesn\u0026rsquo;t work with the package.config file as stated in this issue. The issue says we need to use the SDK style .csproj, but just changing from package.config to Package Reference worked for me. On Visual Studio, click with the right mouse button in References and select Migrate package.config to Package Reference, confirm the changes and we are done: Required keyword, init-only properties and records Here is an example of a .NET Framework 4.7.2 application using the the required keyword, init-only properties and records:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 #nullable enable using System; var person = new Person() { FirstName= \u0026#34;Sherlock\u0026#34;, LastName = \u0026#34;Holmes\u0026#34; }; var address = new Address(\u0026#34;Baker Street 221b\u0026#34;, \u0026#34;London\u0026#34;); Console.WriteLine($\u0026#34;Person: {person.FirstName} {person.LastName}\u0026#34;); Console.WriteLine($\u0026#34;Address: {address}\u0026#34;); Console.ReadKey(); record Address (string StreetName, string City); class Person { public required string FirstName { get; init; } public required string LastName { get; init; } } If we inspect the RequiredMemberAttribute and IsExternalInit, we can see they were generated by PolySharp:\n⚠️ Record types require .NET Framework 4 or superior runtimes. Source code of the examples https://github.com/dgenezini/CSharpNewestFeatures\nReferences and Links PolySharp GitHub Page Sergio Pedri post about PolySharp ","date":"2023-02-06T07:30:00-03:00","dateFormatted":"06 Feb 2023","image":"https://genezini.com/blog/p/how-to-use-c-sharp-11-features-in-dotnet-6-or-older-versions/cover.jpg","permalink":"https://genezini.com/blog/p/how-to-use-c-sharp-11-features-in-dotnet-6-or-older-versions/","title":"How to use C# 11 features in .NET 6 or older versions (even .NET Framework 2.0)"},{"content":"Introduction Integration tests are essential to ensure that the different components of our system work together as expected and continue to work after changes.\nIn this post, I\u0026rsquo;ll explain how to spin up disposable database containers to use in integration tests.\nIntegration tests and managed resources As explained in this article, in integration tests, we should mock unmanaged dependencies (dependencies that are external to our system and not controlled by us, like APIs) but test against real managed dependencies (dependencies that are controlled by our system, like databases). This improves the reliability of the integration tests because the communication with these dependencies are a complex part of the system and can break with a package update, a database update or even a simple change in a SQL command.\nWhat is Testcontainers? Testcontainers is a library that provides lightweight, throwaway instances of databases, selenium web browsers, or anything that can run in a container. These instances can be especially useful for testing applications against real dependencies, like databases, that can be created and disposed of after the tests.\nWhy not run the containers manually? One key benefit of using Testcontainers instead of running the containers manually is that we can make use of libraries such as AutoFixture to generate data to seed the database instead of running scripts to insert the data. This also helps in avoiding collision between data used in different tests because the data is random.\nAlso, there are other advantages in usings Testcontainers:\nTo run the tests locally, you don\u0026rsquo;t need any extra steps, like running a docker-compose command; You don\u0026rsquo;t have to wait or implement a waiting strategy to check if the containers are running before accessing them. Testcontainers already implements this logic; Testcontainers have properties to access the port in which the container is running, so you don\u0026rsquo;t need to specify a fixed port, avoiding port conflict when running in the CI/CD pipeline or other machines; Testcontainers stops and deletes the containers automatically after running, free resources on the machine. Running a disposable container with Testcontainers To use Testcontainers, you will need to have a container runtime (Docker, Podman, Rancher, etc) installed on your machine.\nThen, you need to add the Testcontainers NuGet package to your test project:\ndotnet add package Testcontainers --version 3.1.0 dotnet add package Testcontainers.MySql --version 3.1.0 ⚠️ Since the writing of this post, Testcontainers released version 3.1.0 and made breaking changes. I\u0026rsquo;m updating this post to reflect these changes. To see the original code with Testcontainers 2.3.*, look in this branch of the repository. To run a container, we first need to use the ContainerBuilder class to build a DockerContainer or a derived class, for instance, MySqlBuilder:\n1 2 3 4 5 await using var mySqlContainer = new MySqlBuilder() .WithDatabase(\u0026#34;TestDB\u0026#34;) .WithPassword(\u0026#34;Test1234\u0026#34;) .WithImage(\u0026#34;mysql:8.0.31-oracle\u0026#34;) .Build(); Then, we start the container and use the GetConnectionString method where needed:\n1 2 3 4 5 6 7 8 9 10 await using var mySqlContainer = new MySqlBuilder() .WithDatabase(\u0026#34;TestDB\u0026#34;) .WithPassword(\u0026#34;Test1234\u0026#34;) .WithImage(\u0026#34;mysql:8.0.31-oracle\u0026#34;) .Build(); await mySqlContainer.StartAsync(); await using var todoContext = TodoContext .CreateFromConnectionString(mySqlContainer.GetConnectionString()); 🚨 DockerContainer implements IAsyncDisposable and needs to be disposed of after use. We can use the await using syntax or call the DisposeAsync method. Testcontainers have classes for many different databases (called modules), for example:\nElasticsearch; MariaDB; Microsoft SQL Server; MySQL; Redis. The full list can be seen here.\nBut we can also create containers from any image, as in the example below, where it creates a Memcached instance:\n1 2 3 4 5 await using var MemCachedContainer = new ContainerBuilder() .WithImage(\u0026#34;memcached:1.6.17\u0026#34;) .Build(); await MemCachedContainer.StartAsync(); 💡 In this post, I explain how to use Testcontainers with Localstack to create integration tests for systems that use AWS services. More details in the official documentation.\nCreating integration tests with Testcontainers In this example, I\u0026rsquo;m using the xUnit and the WebApplicationFactory\u0026lt;T\u0026gt; class from ASP.NET Core.\nIf you don\u0026rsquo;t know how to use the WebApplicationFactory\u0026lt;T\u0026gt; class, I explained in this post.\nIn this example, I have a controller with a GET method that returns a ToDo item and a POST method that add a ToDo item:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 [ApiController] [Route(\u0026#34;[controller]\u0026#34;)] public class TodoController : ControllerBase { private readonly TodoContext _todoContext; public TodoController(TodoContext todoContext) { _todoContext = todoContext; } [HttpGet(\u0026#34;{itemId}\u0026#34;, Name = \u0026#34;GetTodoItem\u0026#34;)] public async Task\u0026lt;ActionResult\u0026lt;TodoItem?\u0026gt;\u0026gt; GetByIdAsync(int itemId) { var item = await _todoContext.TodoItems.SingleOrDefaultAsync(a =\u0026gt; a.ItemId == itemId); if (item is null) { return NotFound(); } return Ok(item); } [HttpPost] public async Task\u0026lt;ActionResult\u0026lt;int\u0026gt;\u0026gt; PostAsync(TodoItem todoItem) { _todoContext.Add(todoItem); await _todoContext.SaveChangesAsync(); return CreatedAtRoute(\u0026#34;GetTodoItem\u0026#34;, new { itemId = todoItem.ItemId }, todoItem); } } ⚠️ The business logic is in the controller just for the sake of simplicity. In a real-world application, the logic should be in another layer. Testing the GET method The test does the following actions:\nCreate and start a MySql container; Create a Entity Framework DbContext using the ConnectionString from the MySql in the container; Create the database tables using Entity Framework; (Can also be done passing a script to the ExecScriptAsync method from the mySqlContainer object); Create a random object with AutoFixture and add it to the database table; Override our application configuration with the connection string from the container; Create an HttpClient pointing to our application; Make a request to the GET endpoint passing the Id of the random object we added to the database; Validate that the Status Code is 200 and the object returned is the same we added to the database. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 public class TodoIntegrationTests : IClassFixture\u0026lt;WebApplicationFactory\u0026lt;Program\u0026gt;\u0026gt; { private readonly WebApplicationFactory\u0026lt;Program\u0026gt; _factory; public TodoIntegrationTests(WebApplicationFactory\u0026lt;Program\u0026gt; factory) { _factory = factory; } [Fact] public async Task GetOneItem_Returns200WithItem() { //Arrange await using var mySqlContainer = new MySqlBuilder() .WithDatabase(\u0026#34;TestDB\u0026#34;) .WithPassword(\u0026#34;Test1234\u0026#34;) .WithImage(\u0026#34;mysql:8.0.31-oracle\u0026#34;) .Build(); await mySqlContainer.StartAsync(); await using var todoContext = TodoContext .CreateFromConnectionString(mySqlContainer.GetConnectionString()); // Creates the database if not exists await todoContext.Database.EnsureCreatedAsync(); Fixture fixture = new Fixture(); var todoItem = fixture.Create\u0026lt;TodoItem\u0026gt;(); todoContext.TodoItems.Add(todoItem); await todoContext.SaveChangesAsync(); var HttpClient = _factory .WithWebHostBuilder(builder =\u0026gt; { builder.UseSetting(\u0026#34;MySqlConnectionString\u0026#34;, mySqlTestcontainer.ConnectionString); }) .CreateClient(); //Act var HttpResponse = await HttpClient.GetAsync($\u0026#34;/todo/{todoItem.ItemId}\u0026#34;); //Assert HttpResponse.StatusCode.Should().Be(HttpStatusCode.OK); var responseJson = await HttpResponse.Content.ReadAsStringAsync(); var todoItemResult = JsonSerializer.Deserialize\u0026lt;TodoItem\u0026gt;(responseJson); todoItemResult.Should().BeEquivalentTo(todoItem); } } 🚨 Be aware that I pass the container tag version in the WithImage method even when using the typed MySqlBuilder class. This is very important because when we don\u0026rsquo;t pass the tag, the container runtime will use the latest tag and a database update may break the application and the tests. Testing the POST method First, let\u0026rsquo;s migrate the MySqlContainer and the DbContext creation to a Fixture and a Collection so they can be shared between all tests. This is recommended because unless we do this, all tests will spin up and dispose of the container, making our tests slower than needed.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 [CollectionDefinition(\u0026#34;MySqlTestcontainer Collection\u0026#34;)] public class MySqlTestcontainerCollection: ICollectionFixture\u0026lt;MySqlTestcontainerFixture\u0026gt; { } public class MySqlTestcontainerFixture : IAsyncLifetime { public MySqlContainer MySqlContainer { get; private set; } = default!; public TodoContext TodoContext { get; private set; } = default!; public async Task InitializeAsync() { MySqlContainer = new MySqlBuilder() .WithDatabase(\u0026#34;TestDB\u0026#34;) .WithPassword(\u0026#34;Test1234\u0026#34;) .WithImage(\u0026#34;mysql:8.0.31-oracle\u0026#34;) .Build(); await MySqlContainer.StartAsync(); TodoContext = TodoContext .CreateFromConnectionString(MySqlContainer.GetConnectionString()); // Creates the database if it does not exists await TodoContext.Database.EnsureCreatedAsync(); } public async Task DisposeAsync() { await MySqlContainer.DisposeAsync(); await TodoContext.DisposeAsync(); } } The test does the following actions:\nMySql container and DbContext are injected by the xUnit fixture; Create a random object with AutoFixture; Override our application configuration with the connection string from the container; Create an HttpClient pointing to our application; Make POST request to the endpoint passing the random object previously created; Validate that the Status Code is 200 and that the Location header has the correct URL for the GET endpoint of the created object; Query the database and validate that the object created is equal to the randomly created object. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 [Collection(\u0026#34;MySqlTestcontainer Collection\u0026#34;)] public class TodoIntegrationTests : IClassFixture\u0026lt;WebApplicationFactory\u0026lt;Program\u0026gt;\u0026gt; { private readonly WebApplicationFactory\u0026lt;Program\u0026gt; _factory; private readonly MySqlContainer _mySqlContainer; private readonly TodoContext _todoContext; public TodoIntegrationTests(WebApplicationFactory\u0026lt;Program\u0026gt; factory, MySqlTestcontainerFixture mySqlTestcontainerFixture) { _factory = factory; _mySqlContainer = mySqlTestcontainerFixture.MySqlContainer; _todoContext = mySqlTestcontainerFixture.TodoContext; } //Other tests //... [Fact] public async Task PostOneItem_Returns201AndCreateItem() { //Arrange Fixture fixture = new Fixture(); var todoItem = fixture.Build\u0026lt;TodoItem\u0026gt;() .With(x =\u0026gt; x.ItemId, 0) .Create(); var HttpClient = _factory .WithWebHostBuilder(builder =\u0026gt; { builder.UseSetting(\u0026#34;MySqlConnectionString\u0026#34;, _mySqlContainer.GetConnectionString()); }) .CreateClient(); //Act var HttpResponse = await HttpClient.PostAsJsonAsync($\u0026#34;/todo\u0026#34;, todoItem); //Assert HttpResponse.StatusCode.Should().Be(HttpStatusCode.Created); var responseJson = await HttpResponse.Content.ReadAsStringAsync(); var todoItemResult = JsonSerializer.Deserialize\u0026lt;TodoItem\u0026gt;(responseJson); HttpResponse.Headers.Location.Should().Be($\u0026#34;{HttpClient.BaseAddress}Todo/{todoItemResult!.ItemId}\u0026#34;); var dbItem = await _todoContext.TodoItems .SingleAsync(a =\u0026gt; a.ItemId == todoItemResult!.ItemId); dbItem.Description.Should().Be(todoItem.Description); } } Source code of this sample https://github.com/dgenezini/TestcontainersMySqlSample\n","date":"2023-01-09T07:00:00-03:00","dateFormatted":"09 Jan 2023","image":"https://genezini.com/blog/p/how-to-run-disposable-databases-for-your-tests.-improve-your-integration-tests-accuracy-with-testcontainers/cover.jpg","permalink":"https://genezini.com/blog/p/how-to-run-disposable-databases-for-your-tests.-improve-your-integration-tests-accuracy-with-testcontainers/","title":"How to run disposable databases for your tests. Improve your integration tests accuracy with Testcontainers"},{"content":"An alias is an ordinary config entry: it exists only in the scope where created (git log --graph --oneline → git graph), shell commands included.\nScope Applies to Config file local the current repository .git/config (Windows: .git\\config) global the current user ~/.gitconfig (Windows: %USERPROFILE%\\.gitconfig) system every user on the machine /etc/gitconfig (Windows: %PROGRAMDATA%\\Git\\config) Creating one git config --global alias.c \u0026#34;checkout\u0026#34; git config --global alias.ec \u0026#34;commit --allow-empty -m \u0026#39;Empty commit\u0026#39;\u0026#34; # single quotes when the command has double quotes Or under [alias] in the config file:\n1 2 3 [alias] c = checkout ec = commit --allow-empty -m \u0026#39;Empty commit\u0026#39; Shell aliases An alias starting with ! runs the rest as a shell command (so \u0026amp;\u0026amp; and | work), and must spell git out itself — a shell, not a git subcommand:\ngit config --global alias.search \u0026#34;!f() { git rev-list --all | xargs git grep $1; }; f\u0026#34; ⚠️ Inside a ! alias, always write git explicitly — the shell has no idea it was invoked by git.\nMy aliases Alias Command Usage c checkout git c develop f fetch --prune git f ec commit --allow-empty -m 'Empty commit' git ec graph log --graph --oneline git graph wp worktree prune git wp wa !f() { git worktree prune \u0026amp;\u0026amp; git worktree add $1 $2; }; f git wa ../folder branch-name search !f() { git rev-list --all | xargs git grep $1; }; f git search keyword mb !f() { git branch --show-current | xargs git merge-base $1; }; f git mb origin/main reset-to-base !f() { git mb $1 | xargs git reset; }; f git reset-to-base origin/main mb and reset-to-base exist for one workflow: rebasing a messy branch without resolving one conflict per commit.\nReferences Git Basics — Git Aliases Advanced Git aliases ","date":"2023-01-04T00:00:00Z","dateFormatted":"04 Jan 2023","permalink":"https://genezini.com/garden/git-alias/","title":"Git aliases: scopes, shell escapes, and the ones I use"},{"content":"Each deletes a step: four commands, a branch switch, or a commit you did not mean.\nPrune on fetch --prune drops local refs to remote branches deleted upstream:\ngit fetch --prune Stale refs accumulate with branch churn; git branch -a --contains \u0026lt;commit\u0026gt; gets verbose and slow.\nMerge or rebase without checking out the target A fetch already placed the remote\u0026rsquo;s refs locally as origin/…, and a branch is just a commit reference — no checkout, pull or checkout-back:\ngit fetch --prune git merge origin/branch-to-merge-from The prefix follows the remote name; origin is only the default.\nTrigger a pipeline with an empty commit Instead of appending blank lines to poke CI:\ngit commit --allow-empty -m \u0026#39;Empty commit\u0026#39; git push Read a file from another branch git show main:src/Program.cs git show HEAD~1:src/Program.cs # any ref works, not just a branch No branch switch, no web UI.\nBack to the previous branch git checkout develop git checkout main git checkout - # develop again Search the whole history git rev-list --all | xargs git grep \u0026#34;git\u0026#34; git rev-list --all feeds every commit to git grep, so matches survive even where the line was later deleted — the search alias in the aliases note.\nLog as a graph git log --graph --oneline git log --graph --oneline systemcontext.md # one file --oneline keeps hash and message on one line, making the graph readable at terminal width.\n","date":"2022-12-27T00:00:00Z","dateFormatted":"27 Dec 2022","permalink":"https://genezini.com/garden/git-tips/","title":"Git one-liners: prune, empty commits, show, grep, graph"},{"content":"Introduction When ASP.NET Core is running in an AWS Lambda and receiving requests through an AWS API Gateway, the application is not notified of an API Gateway time-out and keeps processing the request, completing it eventually. This will leave metrics and logs of a successful request when the client received a time-out error.\nIn this post, I\u0026rsquo;ll show how to solve this problem with cancellation tokens and time-outs.\nThe solution The idea is to time-out the request in the ASP.NET application before the API Gateway time-out. This way we can collect logs and metrics of the requests that timed out and have a realistic view of the errors the users are seeing.\nIn this previous post, I\u0026rsquo;ve explained how to cancel requests aborted by the HTTP client using the CancellationToken in the controllers methods.\nThe default ASP.NET\u0026rsquo;s model binding for the CancellationToken injects the RequestAborted property of the HttpContext object in the controller methods. The RequestAborted property holds a CancellationToken that is triggered when the client abandons the request.\nLet\u0026rsquo;s create a middleware to apply the timeout to ASP.NET Core\u0026rsquo;s Cancellation Token.\nFirst, we\u0026rsquo;ll create a new CancellationTokenSource and set a time-out to it.\nThen, we\u0026rsquo;ll use the CancellationToken.CreateLinkedTokenSource method to link the HttpContext.RequestAborted CancellationToken to the new CancellationToken we created with a time-out.\nLastly, we override the HttpContext.RequestAborted Cancellation Token with the token returned by the CreateLinkedTokenSource.\n⚠️ Remember to always dispose of the CancellationTokenSource objects to avoid memory leaks. Use the using keyword or dispose of them in the HttpContext.Response.OnCompleted delegate. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 public class TimeoutCancellationMiddleware { private readonly RequestDelegate _next; private readonly TimeSpan _timeout; public TimeoutCancellationMiddleware(RequestDelegate next, TimeoutCancellationMiddlewareOptions options) { _next = next; _timeout = options.Timeout; } public async Task InvokeAsync(HttpContext context) { //Create a new CancellationTokenSource and set a time-out using var timeoutCancellationTokenSource = new CancellationTokenSource(); timeoutCancellationTokenSource.CancelAfter(_timeout); //Create a new CancellationTokenSource linking the timeoutCancellationToken and ASP.NET\u0026#39;s RequestAborted CancellationToken using var combinedCancellationTokenSource = CancellationTokenSource .CreateLinkedTokenSource(timeoutCancellationTokenSource.Token, context.RequestAborted); //Override the RequestAborted CancellationToken with our combined CancellationToken context.RequestAborted = combinedCancellationTokenSource.Token; await _next(context); } } public class TimeoutCancellationMiddlewareOptions { public TimeSpan Timeout { get; set; } public TimeoutCancellationMiddlewareOptions(TimeSpan timeout) { Timeout = timeout; } } We can also create an extension method to make it easier to configure the middleware:\n1 2 3 4 5 6 7 8 9 public static class TimeoutCancellationMiddlewareExtensions { public static IApplicationBuilder UseTimeoutCancellationToken( this IApplicationBuilder builder, TimeSpan timeout) { return builder.UseMiddleware\u0026lt;TimeoutCancellationMiddleware\u0026gt;( new TimeoutCancellationMiddlewareOptions(timeout)); } } Then, add it to the middleware pipeline in our application:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 public class Program { public static void Main(string[] args) { var builder = WebApplication.CreateBuilder(args); builder.Services.AddControllers(); var app = builder.Build(); ... //Configure a request time-out of 10 seconds app.UseTimeoutCancellationToken(TimeSpan.FromSeconds(10)); ... app.Run(); } } Now, we have to use the CancellationToken injected in the controllers\u0026rsquo; methods and pass it down to all async methods.\nDogImageController:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 [ApiController] [Route(\u0026#34;[controller]\u0026#34;)] public class DogImageController : ControllerBase { private readonly IDogApi _dogApi; private readonly ILogger\u0026lt;DogImageController\u0026gt; _logger; public DogImageController(IDogApi dogApi, ILogger\u0026lt;DogImageController\u0026gt; logger) { _dogApi = dogApi; _logger = logger; } [HttpGet] public async Task\u0026lt;ActionResult\u0026lt;string\u0026gt;\u0026gt; GetAsync(CancellationToken cancellationToken) { try { var dog = await _dogApi.GetRandomDog(cancellationToken); return dog.message; } catch(TaskCanceledException) { _logger.LogError(ex, ex.Message); return StatusCode(StatusCodes.Status500InternalServerError, \u0026#34;Request timed out or canceled\u0026#34;); } } } 🚨 Do not catch TaskCanceledException anywhere except in the controller, or else your API can return an empty 200 / OK result. DogImageUseCase:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 public class DogImageUseCase: IDogImageUseCase { private readonly IDogApi _dogApi; public DogImageUseCase(IDogApi dogApi) { _dogApi = dogApi; } public async Task\u0026lt;string\u0026gt; GetRandomDogImage(CancellationToken cancellationToken) { var dog = await _dogApi.GetRandomDog(cancellationToken); return dog.message; } } Testing the solution To test if it works, let\u0026rsquo;s force a delay in the DogImageUseCase class:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 public class DogImageUseCase: IDogImageUseCase { private readonly IDogApi _dogApi; public DogImageUseCase(IDogApi dogApi) { _dogApi = dogApi; } public async Task\u0026lt;string\u0026gt; GetRandomDogImage(CancellationToken cancellationToken) { var dog = await _dogApi.GetRandomDog(cancellationToken); await Task.Delay(TimeSpan.FromMinutes(1), cancellationToken); return dog.message; } } Creating an integration test for the time-out scenario In this previous post, I explained how to use WireMock.Net to mock API dependencies in integration tests.\nNow, we\u0026rsquo;ll use WireMock.Net to create a mock that responds with a delay. This way, we can validate that our application is stopping the processing of the request when the time-out occurs.\nFirst, we override the HttpClientTimeoutSeconds configuration value to a time-out bigger than our request time-out. This configuration is used to set the Timeout property of the HttpClient. If the HttpClient time-out is smaller than our request time-out, it will abort the request to the dependencies before the application times out.\nThen, we use WireMock\u0026rsquo;s AddGlobalProcessingDelay to insert a delay in the API mock and force our application to time-out before the response.\nLastly, we assert that our application returned the status code 500 with the message Request timed out or canceled.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 public class DogImageTests: IClassFixture\u0026lt;WebApplicationFactory\u0026lt;Program\u0026gt;\u0026gt; { private readonly WebApplicationFactory\u0026lt;Program\u0026gt; _factory; public DogImageTests(WebApplicationFactory\u0026lt;Program\u0026gt; factory) { _factory = factory; } ... [Fact] public async Task Timeout_Returns_500WithMessage() { //Arrange var wireMockSvr = WireMockServer.Start(); var factory = _factory .WithWebHostBuilder(builder =\u0026gt; { builder.UseSetting(\u0026#34;DogApiUrl\u0026#34;, wireMockSvr.Url); //Set the HttpClient timeout to 30, so it doesn\u0026#39;t trigger before our request timeout that is 10 seconds builder.UseSetting(\u0026#34;HttpClientTimeoutSeconds\u0026#34;, \u0026#34;30\u0026#34;); //Set the Request time-out to 5 seconds for the test to run faster builder.UseSetting(\u0026#34;RequestTimeoutSeconds\u0026#34;, \u0026#34;5\u0026#34;); }); var httpClient = factory.CreateClient(); Fixture fixture = new Fixture(); var responseObj = fixture.Create\u0026lt;Dog\u0026gt;(); var responseObjJson = JsonSerializer.Serialize(responseObj); wireMockSvr .Given(Request.Create() .WithPath(\u0026#34;/breeds/image/random\u0026#34;) .UsingGet()) .RespondWith(Response.Create() .WithBody(responseObjJson) .WithHeader(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;) .WithStatusCode(HttpStatusCode.OK)); //Add a delay to the response to cause a request timeout in the /DogImage endpoint wireMockSvr.AddGlobalProcessingDelay(TimeSpan.FromSeconds(15)); //Act var apiHttpResponse = await httpClient.GetAsync(\u0026#34;/DogImage\u0026#34;); //Assert apiHttpResponse.StatusCode.Should().Be(HttpStatusCode.InternalServerError); var responseMessage = await apiHttpResponse.Content.ReadAsStringAsync(); responseMessage.Should().BeEquivalentTo(\u0026#34;Request timed out or canceled\u0026#34;); wireMockSvr.Stop(); } } Complete code Check it on the GitHub Repository.\nReferences and links Integration tests without API dependencies with ASP.NET Core and WireMock.Net Canceling abandoned requests in ASP.NET Core Cancellation in Managed Threads ","date":"2022-12-20T06:45:00-03:00","dateFormatted":"20 Dec 2022","image":"https://genezini.com/blog/p/time-out-requests-in-asp.net-core-with-cancellation-tokens/cover.jpg","permalink":"https://genezini.com/blog/p/time-out-requests-in-asp.net-core-with-cancellation-tokens/","title":"Time-out requests in ASP.NET Core with cancellation tokens"},{"content":"Introduction When a client makes an HTTP request, the client can abort the request, leaving the server processing if it\u0026rsquo;s not prepared to handle this scenario; wasting its resources that could be used to process other jobs.\nIn this post, I\u0026rsquo;ll show how to use Cancellation Tokens to cancel running requests that were aborted by clients.\nCancelation Tokens I\u0026rsquo;ll not explain how Cancelation Tokens work in this post. I recommend reading this great post by Mitesh Shah that explains how they work in a simple way.\nAborted requests There are two main scenarios why requests are aborted by the client:\nWhen loading a page on the browser, a user may click on the stop button to abort the request, or click the refresh button to reload the page.\nWhen a time-out occurs in the client side. A time-out happens when the time that the client is willing to wait for the response expires. When it happens, the client abandons the request and returns and error. It\u0026rsquo;s a good practice to have a time-out configured on the client side when making a request through the network, so it doesn\u0026rsquo;t hang for a long time when the server takes a long time to respond.\nWhy stop processing abandoned requests on the server? First, it\u0026rsquo;s a waste of resources. Memory and CPU that could be used to process other requests are used to process a request that was already discarded by the client. This waste of resources extends to dependencies, like databases and APIs that the system consumes.\nSecond, the abandoned request may slow down other requests, competing for shared resources, like database tables.\n🚨 Be careful what requests you cancel. It may not be a good idea to abort a request that makes changes to the state of the system, even if the requests are idempotent, as it can make the state inconsistent. It may be better to let the request finish. Here are good practices when using Cancellation Tokens. (Extracted from Andrew Arnott\u0026rsquo;s post):\n1 - Know when you’ve passed the point of no cancellation. Don’t cancel if you’ve already incurred side-effects that your method isn’t prepared to revert on the way out that would leave you in an inconsistent state. So if you’ve done some work, and have a lot more to do, and the token is cancelled, you must only cancel when and if you can do so leaving objects in a valid state. This may mean that you have to finish the large amount of work, or undo all your previous work (i.e. revert the side-effects), or find a convenient place that you can stop halfway through but in a valid condition, before then throwing OperationCanceledException. In other words, the caller must be able to recover to a known consistent state after cancelling your work, or realize that cancellation was not responded to and that the caller then must decide whether to accept the work, or revert its successful completion on its own.\n2 - Propagate your CancellationToken to all the methods you call that accept one, except after the “point of no cancellation” referred to in the previous point. In fact if your method mostly orchestrates calls to other methods that themselves take CancellationTokens, you may find that you don’t personally have to call CancellationToken.ThrowIfCancellationRequested() at all, since the async methods you’re calling will generally do it for you.\n3 - Don’t throw OperationCanceledException after you’ve completed the work, just because the token was signaled. Return a successful result and let the caller decide what to do next. The caller can’t assume you’re cancellable at a given point anyway so they have to be prepared for a successful result even upon cancellation.\n4 - Input validation can certainly go ahead of cancellation checks (since that helps highlight bugs in the calling code).\n5 - Consider not checking the token at all if your work is very quick, or you propagate it to the methods you call. That said, calling CancellationToken.ThrowIfCancellationRequested() is pretty lightweight so don’t think too hard about this one unless you see it on perf traces.\n6 - Check CancellationToken.CanBeCanceled when you can do your work more efficiently if you can assume you’ll never be canceled. CanBeCanceled returns false for CancellationToken.None, and in the future possibly for other cases as well.\nCanceling the requests on ASP.NET Core 1 - Add an CancellationToken object as a parameter in the Controller method. ASP.NET will provide the object through model binding for us.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 [ApiController] [Route(\u0026#34;[controller]\u0026#34;)] public class DogImageController : ControllerBase { private readonly IDogImageUseCase _dogImageUseCase; private readonly ILogger\u0026lt;DogImageController\u0026gt; _logger; public DogImageController(IDogImageUseCase dogImageUseCase, ILogger\u0026lt;DogImageController\u0026gt; logger) { _dogImageUseCase = dogImageUseCase; _logger = logger; } [HttpGet] public async Task\u0026lt;ActionResult\u0026lt;string\u0026gt;\u0026gt; GetAsync(CancellationToken cancellationToken) { try { return await _dogImageUseCase.GetRandomDogImage(cancellationToken); } catch(TaskCanceledException ex) { _logger.LogError(ex, ex.Message); return StatusCode(StatusCodes.Status500InternalServerError, \u0026#34;Request cancelled\u0026#34;); } } } 2 - Pass the CancellationToken down to all async methods in your code.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 public class DogImageUseCase: IDogImageUseCase { private readonly IDogApi _dogApi; public DogImageUseCase(IDogApi dogApi) { _dogApi = dogApi; } public async Task\u0026lt;string\u0026gt; GetRandomDogImage(CancellationToken cancellationToken) { var dog = await _dogApi.GetRandomDog(cancellationToken); return dog.message; } } 3 - The CancellationToken will throw an TaskCanceledException when the request is aborted. It\u0026rsquo;s important to log this error to measure how much it is happening.\nCancellationToken with Refit Refit is a .NET library that facilitates consuming REST APIs. It generates a typed client based on an interface. More details here.\nTo pass a CancellationToken to a Refit client as in the example above, just insert a CancellationToken parameter in the interface:\n1 2 3 4 5 public interface IDogApi { [Get(\u0026#34;/breeds/image/random\u0026#34;)] Task\u0026lt;Dog\u0026gt; GetRandomDog(CancellationToken cancellationToken); } ⚠️ By convention, the CancellationToken should be the last parameter in the method. There is even a static analyzer for this rule (CA1068: CancellationToken parameters must come last). More on static analyzers in this post. Testing the solution We can manually test if the solution works by inserting a delay in our code and making a request through the browser to the /DogImage route.\n1 - Insert a 2 minute delay so we have time to stop the request on the client side. It\u0026rsquo;s required to also pass the CancellationToken to the Delay method so it can exit on a request cancellation:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 public class DogImageUseCase: IDogImageUseCase { private readonly IDogApi _dogApi; public DogImageUseCase(IDogApi dogApi) { _dogApi = dogApi; } public async Task\u0026lt;string\u0026gt; GetRandomDogImage(CancellationToken cancellationToken) { var dog = await _dogApi.GetRandomDog(cancellationToken); await Task.Delay(TimeSpan.FromMinutes(2), cancellationToken); return dog.message; } } 2 - Insert a breakpoint inside the catch TaskCanceledException block;\n3 - Run the application and open the route URL in the browser (in this example, /DogImage);\n4 - Hit the browser\u0026rsquo;s stop button or press the esc key.\nWe will hit the breakpoint, confirming the solution is working.\nProblems with this solution AWS API Gateway has a time-out but it does not report the time-out event to the underlying AWS lambda function. If a time-out occurs at the API Gateway, the users will receive a time-out error but the metrics and logs won\u0026rsquo;t show those errors, making it seem like no problems are happening. Some HTTP Client libraries (for example, Python requests and Go net/http) don\u0026rsquo;t have a default timeout, if the consumer doesn\u0026rsquo;t set one, it will wait indefinitely for the server response. This may leave the server processing for a long time in case of a bug, for example. In this post, I explain a complementary solution that mitigates the problems above.\nReferences and links Microsoft\u0026rsquo;s CancellationToken Documentation ASP.NET Core\u0026rsquo;s Model Binding A Deep Dive into C#’s CancellationToken ","date":"2022-12-12T07:30:00-03:00","dateFormatted":"12 Dec 2022","image":"https://genezini.com/blog/p/canceling-abandoned-requests-in-asp.net-core/cover.jpg","permalink":"https://genezini.com/blog/p/canceling-abandoned-requests-in-asp.net-core/","title":"Canceling abandoned requests in ASP.NET Core"},{"content":"Introduction Changing software diagrams is hard. The simple act of adding a new box may require us to drag all the existing boxes and reorganize the diagram. This is one of the main reasons to why software diagrams are constantly left deprecated after the first stages of the development process.\nIn this post I\u0026rsquo;ll show how defining diagrams as code can help in designing and updating software diagrams, and how to automate the process of updating the documentation with those diagrams.\nWhy create diagrams as code? Easy to change: Just change the code and the elements of the diagram are rendered in a good position (sometimes it may need some tweeking); Reuse of code: Components, sprites, and functions can be defined and shared to be used in other diagrams. We can use loops and conditions to make those pieces of code even more reusable. Details here; History of changes: Because it is code, their versions can be tracked and compared with version control systems, like Git, for example; Single style in the whole diagram: Unless explicited, all the elements of the diagram will have the same style, no need to copy style from one element to another or having to resize all boxes after change the size of one; Inclusive: Everybody in the team can checkout the code and change it without fear because changes can be tracked and the style is the same for everyone. PlantUML PlantUML is a highly customizable open-source tool that allow us to create diagrams using code. Despite de name, it supports many types of diagrams besides UML diagrams. It has it\u0026rsquo;s own language and some extensions to other languages like AsciiMath, Creole and LaTeX.\nPlantUML is a Java Command Line tool. We can run it from the command line, but the best experience is with a Visual Studio Code extension.\nRendering from Visual Studio Code There is an extension that integrates PlantUML with Visual Studio Code.\nIt offers syntax hightlighting and a preview of the diagram on the side while editing, and options to export the current or all project diagrams, besides other features.\nAfter installing the extension, open the command pallete and search for PlantUML to see the available options.\nRendering from the command line First, download the compiled JAR from the downloads page or from the GitHub releases page.\nℹ️ You may want to include the path to the plantuml.jar file in the PATH environment variable to be able to use it in any directory. Then, to generate the diagram for one source file, just run the following command:\njava -jar plantuml.jar Sequence.puml We can also generate the diagrams for more than one file using glob patterns:\njava -jar plantuml.jar *.puml C4 Diagrams The C4 model is a different approach to designing software architecture diagrams. I\u0026rsquo;ve talked about it in my previous post.\nPlantUML has native support for C4 Diagrams. We just need to include the library and use its elements.\nHere are some examples:\nSystem Context diagram 1 2 3 4 5 6 7 8 9 10 11 12 13 14 @startuml C4_SystemContext !include \u0026lt;C4/C4_Context\u0026gt; left to right direction Person(user, \u0026#34;User\u0026#34;, \u0026#34;Company employee who has access to the HR system\u0026#34;) System(hrSystem, \u0026#34;HR System\u0026#34;, \u0026#34;Allows users to manage personal data and contract of the company employees\u0026#34;) System_Ext(emailSystem, \u0026#34;E-mail System\u0026#34;, \u0026#34;Responsible for queueing and sending e-mails\u0026#34;) Rel(user, hrSystem, \u0026#34;Create and change employee personal and contract information\u0026#34;, \u0026#34;\u0026#34;) Rel(hrSystem, emailSystem, \u0026#34;Sends notification e-mails using\u0026#34;, \u0026#34;\u0026#34;) @enduml Containers diagram 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 @startuml C4_HRSystem_Containers !define DEVICONS https://raw.githubusercontent.com/tupadr3/plantuml-icon-font-sprites/master/devicons !define AWSPuml https://raw.githubusercontent.com/awslabs/aws-icons-for-plantuml/v14.0/dist !include \u0026lt;C4/C4_Container\u0026gt; !include DEVICONS/msql_server.puml !include DEVICONS/dotnet.puml !include AWSPuml/AWSCommon.puml !include AWSPuml/ApplicationIntegration/SimpleQueueServiceQueue.puml left to right direction Person(user, \u0026#34;User\u0026#34;, \u0026#34;Company employee who has access to the HR system\u0026#34;) System_Boundary(hrSystem, \u0026#34;HR System\u0026#34;) { Container(webApp, \u0026#34;Web Application\u0026#34;, \u0026#34;ASP.NET 7 Application\u0026#34;, \u0026#34;Provides the system functionalities through the web browser\u0026#34;, $sprite=\u0026#34;dotnet\u0026#34;) Container(backgroundService, \u0026#34;E-mail service\u0026#34;, \u0026#34;.NET 7 Application\u0026#34;, \u0026#34;Background service that reads a queue for employee data changes and sends notification e-mails to the employees\u0026#34;, $sprite=\u0026#34;dotnet\u0026#34;) ContainerDb(database, \u0026#34;Database\u0026#34;, \u0026#34;SQL Server 2022\u0026#34;, \u0026#34;Holds employee and contract data\u0026#34;, $sprite=\u0026#34;msql_server\u0026#34;) ContainerQueue(emailQueue, \u0026#34;Queue\u0026#34;, \u0026#34;AWS SQS\u0026#34;, \u0026#34;Holds employee data changes\u0026#34;, $sprite=\u0026#34;SimpleQueueServiceQueue\u0026#34;) } System_Ext(emailSystem, \u0026#34;E-mail System\u0026#34;, \u0026#34;Responsible for queueing and sending e-mails\u0026#34;) Rel(user, webApp, \u0026#34;Create and change employee personal and contract information\u0026#34;, \u0026#34;\u0026#34;) Rel(webApp, database, \u0026#34;Reads / Writes\u0026#34;, \u0026#34;\u0026#34;) Rel(webApp, emailQueue, \u0026#34;Writes notifications to\u0026#34;, \u0026#34;\u0026#34;) Rel(backgroundService, emailQueue, \u0026#34;Reads notifications from\u0026#34;, \u0026#34;\u0026#34;) Rel(backgroundService, database, \u0026#34;Reads employee data from\u0026#34;, \u0026#34;\u0026#34;) Rel(backgroundService, emailSystem, \u0026#34;Sends e-mails using\u0026#34;, \u0026#34;\u0026#34;) @enduml Components diagram 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 @startuml C4_HRSystem_WebApp_Components !define DEVICONS https://raw.githubusercontent.com/tupadr3/plantuml-icon-font-sprites/master/devicons !define AWSPuml https://raw.githubusercontent.com/awslabs/aws-icons-for-plantuml/v14.0/dist !include \u0026lt;C4/C4_Component\u0026gt; !include DEVICONS/msql_server.puml !include AWSPuml/AWSCommon.puml !include AWSPuml/ApplicationIntegration/SimpleQueueServiceQueue.puml left to right direction Container(webApp, \u0026#34;Web Application\u0026#34;, \u0026#34;ASP.NET 7 Application\u0026#34;, \u0026#34;Provides the system functionalities through the web browser\u0026#34;, $sprite=\u0026#34;dotnet\u0026#34;) Container_Boundary(webApp, \u0026#34;Web Application\u0026#34;) { Component(employeesController, \u0026#34;Employees Controller\u0026#34;, \u0026#34;Provides access to the employees related functionalities\u0026#34;) Component(registerEmployeesUseCase, \u0026#34;Register Employee Use Case\u0026#34;, \u0026#34;Orchestrate the use case of registering a new employee\u0026#34;) Component(employeeDataQueueService, \u0026#34;Employee Data Queue Service\u0026#34;, \u0026#34;Provides functionalities to communicate with the queue\u0026#34;) Component(employeeRepository, \u0026#34;Employee Repository\u0026#34;, \u0026#34;Provides functionalities to communicate with the employee database table\u0026#34;) Component(loginController, \u0026#34;Login Controller\u0026#34;, \u0026#34;ASP.NET Core Controller\u0026#34;, \u0026#34;Allow users to authenticate in the web application\u0026#34;) Rel(employeesController, registerEmployeesUseCase, \u0026#34;Uses\u0026#34;) Rel(registerEmployeesUseCase, employeeDataQueueService, \u0026#34;Uses\u0026#34;) Rel(registerEmployeesUseCase, employeeRepository, \u0026#34;Uses\u0026#34;) } ContainerDb(database, \u0026#34;Database\u0026#34;, \u0026#34;SQL Server 2022\u0026#34;, \u0026#34;Holds employee and contract data\u0026#34;, $sprite=\u0026#34;msql_server\u0026#34;) ContainerQueue(emailQueue, \u0026#34;Queue\u0026#34;, \u0026#34;AWS SQS\u0026#34;, \u0026#34;Holds employee data changes\u0026#34;, $sprite=\u0026#34;SimpleQueueServiceQueue\u0026#34;) Rel(employeeRepository, database, \u0026#34;Writes employee information\u0026#34;, \u0026#34;\u0026#34;) Rel(employeeDataQueueService, emailQueue, \u0026#34;Writes notifications to\u0026#34;, \u0026#34;\u0026#34;) @enduml More examples Here are more examples of what can be done with PlantUML.\nVariables and colors In this example, I created a sequence diagram and used variables for reusing the formatted HTTP verbs in the messages:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 @startuml SequenceDiagram !$get_method = \u0026#34;\u0026lt;font color=lime\u0026gt;\u0026lt;b\u0026gt;GET\u0026lt;/b\u0026gt;\u0026lt;/font\u0026gt;\u0026#34; !$post_method = \u0026#34;\u0026lt;font color=blue\u0026gt;\u0026lt;b\u0026gt;POST\u0026lt;/b\u0026gt;\u0026lt;/font\u0026gt;\u0026#34; participant \u0026#34;Frontend\u0026#34; as Frontend participant \u0026#34;BFF\u0026#34; as BFF participant \u0026#34;PokéAPI\u0026#34; as PokeAPI database \u0026#34;Cache\u0026#34; as Cache Frontend -\u0026gt; BFF : $get_method /pokemondata/{name} BFF -\u0026gt; Cache : $get_method Search for data in the cache BFF \u0026lt;-- Cache : Cached data alt data not found in cache BFF -\u0026gt; PokeAPI : $get_method /pokemon/{name} BFF \u0026lt;-- PokeAPI : Pokemon data BFF -\u0026gt; Cache : $post_method Save data in the cache end Frontend \u0026lt;-- BFF : Return pokémon data @enduml Visualization of JSON data One cool diagram that PlantUML can generate is the JSON diagram, showing the properties and data of a JSON. To generate a JSON diagram, just use the @startjson and @endjson symbols and paste the JSON content between them.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 @startjson JSONDiagram { \u0026#34;abilities\u0026#34;: [ { \u0026#34;ability\u0026#34;: { \u0026#34;name\u0026#34;: \u0026#34;blaze\u0026#34;, \u0026#34;url\u0026#34;: \u0026#34;https://pokeapi.co/api/v2/ability/66/\u0026#34; }, \u0026#34;is_hidden\u0026#34;: false, \u0026#34;slot\u0026#34;: 1 }, { \u0026#34;ability\u0026#34;: { \u0026#34;name\u0026#34;: \u0026#34;solar-power\u0026#34;, \u0026#34;url\u0026#34;: \u0026#34;https://pokeapi.co/api/v2/ability/94/\u0026#34; }, \u0026#34;is_hidden\u0026#34;: true, \u0026#34;slot\u0026#34;: 3 } ], \u0026#34;base_experience\u0026#34;: 267, \u0026#34;forms\u0026#34;: [ { \u0026#34;name\u0026#34;: \u0026#34;charizard\u0026#34;, \u0026#34;url\u0026#34;: \u0026#34;https://pokeapi.co/api/v2/pokemon-form/6/\u0026#34; } ], \u0026#34;height\u0026#34;: 17, \u0026#34;held_items\u0026#34;: [], \u0026#34;id\u0026#34;: 6, \u0026#34;is_default\u0026#34;: true, \u0026#34;location_area_encounters\u0026#34;: \u0026#34;https://pokeapi.co/api/v2/pokemon/6/encounters\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;charizard\u0026#34;, \u0026#34;order\u0026#34;: 7, \u0026#34;past_types\u0026#34;: [], \u0026#34;species\u0026#34;: { \u0026#34;name\u0026#34;: \u0026#34;charizard\u0026#34;, \u0026#34;url\u0026#34;: \u0026#34;https://pokeapi.co/api/v2/pokemon-species/6/\u0026#34; }, \u0026#34;types\u0026#34;: [ { \u0026#34;slot\u0026#34;: 1, \u0026#34;type\u0026#34;: { \u0026#34;name\u0026#34;: \u0026#34;fire\u0026#34;, \u0026#34;url\u0026#34;: \u0026#34;https://pokeapi.co/api/v2/type/10/\u0026#34; } }, { \u0026#34;slot\u0026#34;: 2, \u0026#34;type\u0026#34;: { \u0026#34;name\u0026#34;: \u0026#34;flying\u0026#34;, \u0026#34;url\u0026#34;: \u0026#34;https://pokeapi.co/api/v2/type/3/\u0026#34; } } ], \u0026#34;weight\u0026#34;: 905 } @endjson Importing custom elements PlantUML is extensible, so we can create or import custom elements to use in our diagrams. One example is the AWS Icons for PlantUML. It has elements to represent most of the main AWS Services.\nTo use it, we need to import the custom elements with the !import command.\nIn this example, I used the !define command to define a variable AWSPuml with the base URL and used it in all imports. This helps when we need to update the version of the objects and also makes the code cleaner.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 @startuml InfrastructureDiagram left to right direction !define AWSPuml https://raw.githubusercontent.com/awslabs/aws-icons-for-plantuml/v14.0/dist !include AWSPuml/AWSCommon.puml !include AWSPuml/NetworkingContentDelivery/CloudFront.puml !include AWSPuml/Compute/Lambda.puml !include AWSPuml/Storage/SimpleStorageService.puml !include AWSPuml/Database/ElastiCache.puml actor \u0026#34;User\u0026#34; as User CloudFront(CloudFront, \u0026#34;CloudFront\u0026#34;, \u0026#34;\u0026#34;) SimpleStorageService(S3, \u0026#34;S3 Bucket\u0026#34;, \u0026#34;Angular App\u0026#34;) Lambda(Bff, \u0026#34;BFF\u0026#34;, \u0026#34;ASP.NET Core\u0026#34;) ElastiCache(Redis, \u0026#34;Cache\u0026#34;, \u0026#34;Redis\u0026#34;) User --\u0026gt; CloudFront CloudFront --\u0026gt; S3 S3 --\u0026gt; Bff Bff --\u0026gt; Redis @enduml Themes PlantUML support some themes by default. Just use the !theme command followed by the theme name:\n1 2 3 4 @startuml !theme materia ... @enduml Here is the previous sequence diagram with the Materia theme:\nHere we can see a gallery with the available themes.\nAutomating the diagram publication I\u0026rsquo;ve created a documentation site as an example of how to generate PlantUML diagrams in the CI/CD pipeline. When the diagrams are commited to the repository, the pipeline renders then as images and the documentation is automatically updated.\nhttps://dgenezini.github.io/docs-sample/\nIt uses:\nGitHub Pages for Hosting; GitHub Actions for CI/CD Pipeline (Generate diagrams and deploy static site); Generate PlantUML GitHub Action to generate the diagrams and commit them to the repository; Just the docs template as template for the site. To render the diagrams as images and commit then to repository, configure a new job generate_plantuml with the code below:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 jobs: generate_plantuml: runs-on: ubuntu-latest name: plantuml steps: - name: checkout uses: actions/checkout@v1 with: fetch-depth: 1 - name: plantuml id: plantuml uses: grassedge/generate-plantuml-action@v1.5 with: path: diagrams message: \u0026#34;Render PlantUML files\u0026#34; env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} Then, just reference the images in the documentation files:\n![](/docs-sample/diagrams/2-containers.svg) The full pipeline configuration can be seen here.\nThe source repository is here.\nDiagram as code 2.0 Simon Brown, creator of the C4 model, has a very interesting concept called Diagram as code 2.0 in which one model code can generate multiple diagrams. More details in his blog post.\nHe built a tool called Structurizr for this.\nReferences and Links https://plantuml.com/ https://marketplace.visualstudio.com/items?itemName=jebbs.plantuml https://github.com/plantuml-stdlib/C4-PlantUML https://github.com/awslabs/aws-icons-for-plantuml https://github.com/marketplace/actions/generate-plantuml https://just-the-docs.github.io/just-the-docs/ ","date":"2022-11-29T08:20:00-03:00","dateFormatted":"29 Nov 2022","image":"https://genezini.com/blog/p/keep-your-diagrams-updated-with-continuous-delivery/cover.jpg","permalink":"https://genezini.com/blog/p/keep-your-diagrams-updated-with-continuous-delivery/","title":"Keep your diagrams updated with continuous delivery"},{"content":"Introduction Diagrams are a great way to visually communicate something. However, the majority of software architecture diagrams don\u0026rsquo;t really express what they intend to, lacking description of its elements and with lots of implicit notations only understood by who drew the diagram. Also, they try to express more than one vision (high level, infrastructure, flow order, etc) in only one diagram, making it even harder for non technical people or people outside the project to understand.\nIn this post, I\u0026rsquo;ll present the C4 model and how it can produce software architecture diagrams that are inclusive to non-technical people, easier to understand and, consequently, better to document our software systems.\nThe C4 model The C4 model is a different approach to designing software architecture diagrams. It has a concept of four main hierarchical levels of diagrams and prioritizes abstractions over notation. What this means is that common types of elements are more important than types of boxes and types of arrows with implicit meanings. Notations cause confusion and limit the people who can read the diagrams to those who know the notation or the project.\nThe abstractions exist to help those who draw the diagrams, in contrast to notations that need to be known by everybody who reads the diagrams. The idea is that the diagrams should be understood by themselves, without any prior knowledge of the project or the notation. It should assist the communication inside and outside the team.\nIt also has a clear definition of the scope of each diagram, with complementary diagrams, for example, for showing the sequence of events, or the infrastructure where the system is deployed at.\nMain diagrams The levels of diagrams work as a map, where we zoom in from country to city to see more details.\nIn the image below we zoom in on the system to see its containers, zoom in on a container to see its components, and zoom in on a component to see its code.\nSystem Context The first level diagram gives a context of how the system interacts with the users and other external systems. Details about technology should not be on this level and it should be understood by non-technical people.\nHere we can see what is in the scope of our system and what is not. In addition, we see the interactions of our system with users and external systems.\nElements in this diagram:\nThe system in scope; Users/people and their interactions with the system; External systems and their interactions with the system. Containers The containers diagram shows the containers inside the system boundary. Containers in the C4 model have no relation to docker and other container runtimes; they are just a name for a part of the system. Containers are almost always separately deployable applications that compose a system and that communicate with each other through inter-process communication or via network. Examples of containers are: single-page applications, desktop applications, APIs, databases, queues, event topics, and storage paths.\nAt this level, we should start describing the technology used in the containers and in the interactions. For instance, \u0026ldquo;Mobile App - [Container: Xamarin/C#]\u0026rdquo;, \u0026ldquo;Makes API calls to [JSON/HTTPs]\u0026rdquo;.\nIt still shows the users and external systems, but in this level, it shows the interactions with each container. In the diagram below, the user interacts with the Single-Page Application, and not with the Internet Banking System boundary.\nElements in this diagram:\nThe system boundary; Users/people and their interactions with the containers; External systems and their interactions with the containers. Components The components diagram shows one container broken down into components and their interactions with each other and other containers or external systems. Components are important units of code; they may be interface elements in frontend apps, controllers in APIs or pages in web apps.\nThis level is recommended only when judged useful because the more we zoom in, the harder it is to maintain it updated.\nElements in this diagram:\nThe container boundary; Other containers and their interactions with the components of the container in scope; External systems and their interactions with the components. Code In the code diagram, we can give details of the components with UML class diagrams or other similar diagrams.\nThis level is not recommended because it is very difficult to keep updated unless automatically generated from the source code. Even then, it should be used only when strictly necessary.\nElements in this diagram:\nThe component boundary; Classes and interfaces within the component. Supplementary diagrams C4 has more diagrams that can be used when necessary:\nSystem Landscape diagram One level above the system context diagram. Useful if showing more than one system is needed.\nDynamic diagram Similar to the UML Sequence diagram. Shows the order of the interactions between the elements.\nPersonally, I find the UML sequence Diagram easier to understand even for non-technical people, because they have an order from top to bottom and from left to right, so I normally use it instead of the C4 dynamic diagram.\nDeployment diagram Describes the infrastructure and how the containers of the system are deployed in it.\nHere are more details about these diagrams.\nSome important notes Diagrams should have title and their type on the header; Acronyms should be avoided to make the diagrams inclusive for everybody; Elements should have their type and the description of their responsibilities; Interactions between elements should have one and only one direction (no lines with zero or two directions) and be descriptive. Example: \u0026ldquo;sends emails using\u0026rdquo; instead of \u0026ldquo;uses\u0026rdquo;; Interactions don\u0026rsquo;t need a response line. Two lines may be used only if the interaction may start from both elements; The direction of the interaction doesn\u0026rsquo;t matter as long as the description matches it; Types of boxes and color should be complementary and not required for understanding the diagram. Diagram review checklist There is a checklist tool we can use to review our diagrams here.\nDiagram as code It is highly recommended to create C4 diagrams as code. It makes them easier to change and maintain updated. I\u0026rsquo;ve talked about diagrams as code in this post.\nReferences and Links https://c4model.com https://c4model.com/review/ ","date":"2022-11-22T05:45:00-03:00","dateFormatted":"22 Nov 2022","image":"https://genezini.com/blog/p/cleaner-representation-of-software-architectures-with-the-c4-model/cover.jpg","permalink":"https://genezini.com/blog/p/cleaner-representation-of-software-architectures-with-the-c4-model/","title":"Cleaner representation of software architectures with the C4 Model"},{"content":"Introduction Automated software tests are a requirement for ensuring we are delivering a product with quality to our users. It helps in finding bugs and requirements not fulfilled at development time, but also decreases the cost of maintenance by making the future changes to our code safer. Besides, the act of writing testable code alone increases the quality of the code we are writing because testable code has to be decoupled.\nIn this last post of this series, I\u0026rsquo;ll show how to analyze and enforce a minimum code coverage in our applications, and how to use integration tests to increase our testing surface.\nWhat is code coverage? Code coverage is a software metric that shows how much of our code is executed (covered) by our automated tests. It is shown as a percentage and can be calculated with different formulas, based on the number of lines or branches, for example. The higher the percentage, more of our code is being tested.\nAnalyzing the code coverage of our application In this example, we have an ASP.NET Core API with a simple use case class that checks an input number and returns a string telling if the number is even or odd:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 using CodeCoverageSample.Interfaces; namespace CodeCoverageSample.UseCases; public class IsEvenUseCase : IIsEvenUseCase { public string IsEven(int number) { if (number % 2 == 0) { return \u0026#34;Number is even\u0026#34;; } else { return \u0026#34;Number is odd\u0026#34;; } } } For now, we have only one unit test for this use case class:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 using CodeCoverageSample.UseCases; namespace CodeCoverageSample.UnitTests; public class IsEvenUseCaseTests { [Fact] public void EvenNumber_ReturnsEven() { //Arrange var isEvenUseCase = new IsEvenUseCase(); //Act var result = isEvenUseCase.IsEven(2); //Assert Assert.Equal(\u0026#34;Number is even\u0026#34;, result); } } To analyze the code coverage of our application, first we need to install Coverlet\u0026rsquo;s MSBuild integration using the coverlet.msbuild nuget package in our test project:\nInstall-Package coverlet.msbuild Then, run the dotnet test command with the Coverlet options on the solution or project folder:\ndotnet test -p:CollectCoverage=true -p:CoverletOutputFormat=opencover -p:CoverletOutput=../TestResults We are using the following options:\nCollectCoverage: Inform dotnet test to use coverlet to collect the code coverage data; CoverletOutputFormat: The format of the report that coverlet will generate (opencover, cobertura, json). More here; CoverletOutput: The path where the coverage report will be saved in. This path is relative to the test project; This will print the code coverage result in a table and generate a report file named TestResults.opencover.xml:\n⚠️ We can also run coverlet on the command line with the coverlet.collector nuget package, but it has limited options and doesn\u0026rsquo;t print the results in the command line. More details here; Generating HTML reports Coverlet generates the report in formats that are not easily readable by humans, so we need to generate an HTML report based on Coverlet report. To do it, we\u0026rsquo;ll usa a tool called ReportGenerator.\nInstalling ReportGenerator ReportGenerator is installed as a .NET global tool.\nTo do this, we run the following command:\ndotnet tool install --global dotnet-reportgenerator-globaltool --version 4.8.6 Generating an HTML report of the opencover report To generate an HTML report based on a Coverlet report, we run the following command:\nreportgenerator \u0026#34;-reports:TestResults.opencover.xml\u0026#34; \u0026#34;-targetdir:coveragereport\u0026#34; -reporttypes:Html We are using the following options:\nreports: The path to the coverage report; targetdir: The path where the HTML report will be saved in; reporttypes: The format the report will be generated in. The command output will tell the relative path to the generated report: coveragereport\\index.html.\nOpening the coveragereport\\index.html file we can see the project Line and Branch coverage:\nClicking on CodeCoverageSample.UseCases.IsEvenUseCase we can see details of the code coverage by method (in the table) and the line and branch coverage for the class:\nLine vs Branch coverage But what is line coverage and branch coverage?\nLine coverage: Indicates the percentage of lines that are covered by the tests; Branch coverage: Indicates the percentage of logical paths that are covered by the tests (if, else, switch condition, etc). In the example below, we can see that two lines in the else branch are not covered by the tests.\nThis will result in:\n50% of branch coverage, because only the if branch is covered; 71.4% of line coverage, because only 5 of the 7 lines are covered. Code coverage on Visual Studio The is an extension called Run Coverlet Report that integrates Coverlet and ReportGenerator with Visual Studio.\nFirst, we need to install the coverlet.collector nuget package in our test projects. Xunit template already has this package installed by default. Install-Package coverlet.collector Then, navigate to Extensions \u0026gt; Manage extensions and install the Run Coverlet Report extension. Navigate to the new option Tools \u0026gt; Run Code Coverage. This will generate the ReportGenerator HTML report that will be open in Visual Studio. Also, after running the code coverage tool, Visual studio will read the coverlet report and show the coverage in our source file:\nImproving our code coverage Fixing the unit tests Now we will implement the OddNumber_ReturnsOdd method to test the logical path we didn\u0026rsquo;t test before:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 using CodeCoverageSample.UseCases; namespace CodeCoverageSample.UnitTests; public class IsEvenUseCaseTests { [Fact] public void EvenNumber_ReturnsEven() { //Arrange var isEvenUseCase = new IsEvenUseCase(); //Act var result = isEvenUseCase.IsEven(2); //Assert Assert.Equal(\u0026#34;Number is even\u0026#34;, result); } [Fact] public void OddNumber_ReturnsOdd() { //Arrange var isEvenUseCase = new IsEvenUseCase(); //Act var result = isEvenUseCase.IsEven(3); //Assert Assert.Equal(\u0026#34;Number is odd\u0026#34;, result); } } This will increase our average coverage to 50% of branch and 22.58% of line:\nAnd 100% for the IsEvenUseCase class:\nImplementing integration tests Integration tests using the WebApplicationFactory class (More here) are also considered in the code coverage reports. Let\u0026rsquo;s look at our IsEvenController and Program classes coverage:\nLet\u0026rsquo;s implement a simple integration test. It will just instantiate our API and make a call passing a number and validate the results:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 using Microsoft.AspNetCore.Mvc.Testing; using System.Net; namespace CodeCoverageSample.UnitTests.IntegrationTests; public class IsEvenIntegrationTest : IClassFixture\u0026lt;WebApplicationFactory\u0026lt;Program\u0026gt;\u0026gt; { private readonly WebApplicationFactory\u0026lt;Program\u0026gt; _factory; public IsEvenIntegrationTest(WebApplicationFactory\u0026lt;Program\u0026gt; factory) { _factory = factory; } [Theory] [InlineData(2, \u0026#34;Number is even\u0026#34;)] [InlineData(3, \u0026#34;Number is odd\u0026#34;)] public async Task Number_ReturnsCorrectAndOk(int number, string expectedResult) { var HttpClient = _factory .CreateClient(); //Act var HttpResponse = await HttpClient.GetAsync($\u0026#34;/iseven/{number}\u0026#34;); //Assert Assert.Equal(HttpStatusCode.OK, HttpResponse.StatusCode); var ResponseStr = await HttpResponse.Content.ReadAsStringAsync(); Assert.Equal(expectedResult, ResponseStr); } } Now we run the code coverage report again and the IsEvenController and Program classes are covered by the tests:\nRemoving code from the code coverage analysis If we want to remove a class or method from the code coverage analysis, we can decorate it with the ExcludeFromCodeCoverage attribute:\n1 2 3 4 5 6 7 8 9 using System.Diagnostics.CodeAnalysis; namespace CodeCoverageSample; [ExcludeFromCodeCoverage] public class DoNotTestMe { ... } ℹ️ We can also create custom attributes to exclude from coverlet code coverage. Details here. Ignoring auto-properties Coverlet has the SkipAutoProps option to exclude the auto-properties from the coverage report.\nFor example, this class doesn\u0026rsquo;t have any logic and doesn\u0026rsquo;t need that the get and set methods of its properties be tested:\n1 2 3 4 5 6 7 namespace CodeCoverageSample; public class NoLogicHere { public int Id { get; set; } public int Name { get; set; } } Just set the SkipAutoProps to true when running the code coverage from the command line:\ndotnet test -p:CollectCoverage=true -p:CoverletOutputFormat=opencover -p:CoverletOutput=TestResults -p:SkipAutoProps=true ⚠️ Unfortunately, the Run Coverage Report extension still doesn\u0026rsquo;t allow us to configure the coverlet parameters. There is an open pull request with this feature awaiting for approval here. Enforcing a minimum code coverage on the build pipeline Just like code style and code quality rules, that I talked about in my previous post, we need to enforce a minimum code coverage in our build pipeline to maintain a level of quality in our code. Coverlet has the Threshold option that we can set to a minimum percentage and it will fail the tests if our code coverage is below this percentage:\ndotnet test -p:CollectCoverage=true -p:CoverletOutputFormat=opencover -p:CoverletOutput=TestResults -p:SkipAutoProps=true -p:Threshold=80 We can also use the ThresholdType option to set the type of coverage to enforce. Not specifying will enforce all types of coverage (Line, Branch and Method). Details here.\nReferences and Links https://github.com/coverlet-coverage/coverlet https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/MSBuildIntegration.md https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/VSTestIntegration.md https://github.com/danielpalme/ReportGenerator https://marketplace.visualstudio.com/items?itemName=ChrisDexter.RunCoverletReport https://github.com/the-dext/RunCoverletReport/blob/master/README.md ","date":"2022-11-03T08:10:00-03:00","dateFormatted":"03 Nov 2022","image":"https://genezini.com/blog/p/analyzing-and-enforcing-.net-code-coverage-with-coverlet/cover.png","permalink":"https://genezini.com/blog/p/analyzing-and-enforcing-.net-code-coverage-with-coverlet/","title":"Analyzing and enforcing .NET code coverage with coverlet"},{"content":"Introduction Static code analysis is a great tool for spotting some kinds of error in your code, for example, not disposing of objects that implement IDisposable. Also, it helps to enforce and validate if the code written is following a defined standard, for example, using PascalCase for class names and camelCase for parameter names.\nIn this post I\u0026rsquo;ll show how to use Roslyn Analyzers with C# to enforce some standards of code quality and code style on your code, throwing errors at compile time if any rules are not being respected and not allowing the code to be pushed to protected branches of the repository.\nRoslyn Analyzers Roslyn is the compiler platform for .NET. Roslyn Analyzers are static code analysis tools for Roslyn. They inspect your code for style, quality, maintainability, and practices that are likely to cause bugs. They work based on predefined rules that can have their severity configured in the EditorConfig file.\n.NET 5 and later have the analyzers enabled by default. To enable them in earlier versions of .NET, you can set the property EnableNETAnalyzers to true on project files that uses a project SDK or install them as a nuget package:\nSetting EnableNETAnalyzers on the project file 1 2 3 \u0026lt;PropertyGroup\u0026gt; \u0026lt;EnableNETAnalyzers\u0026gt;true\u0026lt;/EnableNETAnalyzers\u0026gt; \u0026lt;/PropertyGroup\u0026gt; Installing as a nuget package Install-Package Microsoft.CodeAnalysis.NetAnalyzers Enabling more analyzers By default, only some rules are enabled, but we can configure this with the AnalysisMode property in the project file:\n1 2 3 \u0026lt;PropertyGroup\u0026gt; \u0026lt;AnalysisMode\u0026gt;Recommended\u0026lt;/AnalysisMode\u0026gt; \u0026lt;/PropertyGroup\u0026gt; The AnalysisMode property values allowed are different for .NET 6 and .NET 5 SDKs. Details here.\nHow to enable .NET Analyzers in VS Code .NET Analyzers work by default in Visual Studio, but they have to be enabled in VS Code.\n1 - Navigate to File \u0026gt; Preferences \u0026gt; Settings.\n2 - Navigate to Extensions \u0026gt; C# configuration or search for omnisharp.enableRoslynAnalyzers.\n3 - Check the Omnisharp: Enable Roslyn Analyzers option.\n4 - Navigate to Extensions \u0026gt; C# configuration or search for omnisharp.enableEditorConfigSupport.\n5 - Check the Omnisharp: Enable Editor Config Support option.\n6 - Restart C#/Omnisharp extension or VS Code.\nTypes of rules .NET Analyzers have many categories of rules, but here I\u0026rsquo;ll list just a few to explain how they interact with Visual Studio\u0026rsquo;s features.\nStandard formatting: Default Editorconfig options, like indent size and tabs or spaces;\nCode Style - .NET Formatting: Language specific indentation, whitespaces, and wrapping. For instance, use spaces before parentheses in method definitions.\nCode Style - .NET Language: C# and Visual Basic specific rules. Examples: using var instead of an explicit type, prefer auto properties instead of backing fields.\nCode Style - Naming Conventions: Rules about the naming of code elements, like enforcing PascalCase for classes\u0026rsquo; names and Async at the end of async methods\u0026rsquo; names.\nCode Style - Unnecessary code: Rules for code that is unreachable or unused variables, fields, etc.\nCode Quality: Rules to improve code quality. These rules help identify code that are likely to cause bugs or security problems. Examples: Do not declare static members on generic types, and Enums should have zero value.\nThe table below shows in which features of Visual Studio the fixes for these types of rules are applied on.\nFixes applied on 🖹 Format 🧹 Code Cleanup 💡 Code Fix Standard Formatting ✔️ ✔️ ✔️ .NET Formatting ✔️ ✔️ ✔️ .NET Language ✔️ ✔️ Naming Conventions ✔️ Unnecessary Code ❗ ✔️ Code Quality ❗ ❗ Only some rules have fixes applied. 💡 In the previous post of this series, I explain how to configure Visual Studio to apply this rules on Code Cleanup and how to auto execute Code Cleanup on file save. Enforcing rules in our code Rules are configured in the EditorConfig file (that I explained in a Part 1 of this series) and their severity can be defined in three levels. Conflicts in the rules are resolved in the following order:\nSpecific rules Category rules All analyzers rules In the example below, Naming rules violations (IDE1006) will be considered Warning, because it is defined for the specific rule:\n1 2 3 4 5 6 # Defines that all analyzers rules are suggestions dotnet_analyzer_diagnostic.severity = suggestion # Defines that all Code Style analyzers rules are errors dotnet_analyzer_diagnostic.category-Style.severity = error # Defines that the rule IDE1006 is a warning dotnet_diagnostic.IDE1006.severity = warning 1. Generate an EditorConfig file from Visual Studio First, we need to create an EditorConfig file with the configuration of the rules we will use as standards.\nVisual Studio has a tool to help you configure the code style rules of your EditorConfig file, showing a snippet of code of how the rules work.\nGo to Tools \u0026gt; Options \u0026gt; Text Editor \u0026gt; C# \u0026gt; Code Style; Configure your Code Style preferences in the General, Formatting and Naming sub-menus. ⚠️ Don\u0026rsquo;t bother setting the severities here; some of them are only respected by Visual Studio and are not enforced on build and other IDEs; Back in the General sub-menu, click Generate .editoconfig file from settings and save it in the folder your solution file is in (.sln). ⚠️ If you are not using Visual Studio, you can use a sample and change it to your preferences, like the one from Roslyn. 2. Configure all projects to use the recommended .NET Analyzers Next, we set the AnalysisMode property in all our project files.\nFor .NET 6 SDK and later, set it to Recommended or All.\n1 2 3 \u0026lt;PropertyGroup\u0026gt; \u0026lt;AnalysisMode\u0026gt;Recommended\u0026lt;/AnalysisMode\u0026gt; \u0026lt;/PropertyGroup\u0026gt; 3. Set severity Error for all analyzers rules In our EditorConfig, include this line to set severity to error for all rules.\n# Set severity = error for all analyzers dotnet_analyzer_diagnostic.severity = error 4. Correct the errors and override the severity for rules you don\u0026rsquo;t want to use If you are enabling the analyzers in an existing project, many errors will be shown. Correct them and override their severity if they don\u0026rsquo;t apply for you or you won\u0026rsquo;t correct them at the moment.\n💡 In the previous post of this series, I explain how to add fixers to Visual Studio\u0026rsquo;s Code Cleanup. You can customize it to fix some rules violations. Setting rules severity directly in EditorConfig file 1 2 3 4 5 6 7 # Other rules ... # Set severity = none to the rules that are not important for me dotnet_diagnostic.IDE0075.severity = none # Set severity = warning to the rules that need to be resolved later dotnet_diagnostic.IDE0047.severity = warning Setting rules severity from Visual Studio\u0026rsquo;s Error List For errors showing up in the Error List, you can right click on the rule and click on Set severity \u0026gt; Choose a severity. The severity configuration will be added to the EditorConfig file.\nSetting rules severity from Visual Studio\u0026rsquo;s Solution Explorer From Solution Explorer, open the Dependencies \u0026gt; Analyzers node below your project, then right click on the rule and click on Set severity \u0026gt; Choose a severity. The severity configuration will be added to the EditorConfig file.\n5. Enforce the rules on build Enabling the analyzers only shows the messages in our IDE. To really enforce those rules, we have to inform the compiler to fail in case of rules violations, blocking changes that are not compliant to the standard to be merged into protected branches of the repository.\nTo do this, we need to enable the property EnforceCodeStyleInBuild in all our project files.\n1 2 3 \u0026lt;PropertyGroup\u0026gt; \u0026lt;EnforceCodeStyleInBuild\u0026gt;true\u0026lt;/EnforceCodeStyleInBuild\u0026gt; \u0026lt;/PropertyGroup\u0026gt; Examples of rules being enforced Rules being enforced on Visual Studio Rules being enforced on VS Code Rules being enforced on dotnet build command Creating additional naming conventions Here are some naming conventions of the C# language:\nSymbols Convention Example class/record/struct PascalCase PhysicalAddress interface \u0026ldquo;I\u0026rdquo;+PascalCase IWorkerQueue public members PascalCase StartEventProcessing private/internal fields \u0026ldquo;_\u0026quot;+camelCase _workerQueue static fields \u0026ldquo;s_\u0026quot;+camelCase s_workerQueue local variables *️ camelCase isValid parameters camelCase name async methods PascalCase+\u0026ldquo;Async\u0026rdquo; GetStringAsync More details here.\nBy default, Visual Studio doesn\u0026rsquo;t create naming conventions for static fields, local variables, parameters and async methods. If we want to use them, we have to manually set those rules, as shown below.\n*️ Not specified in the docs, but Roslyn uses this convention.\nCreating the naming convention for async methods 1 2 3 4 5 6 7 8 9 10 dotnet_naming_rule.async_methods_should_be_pascalcase_async.severity = error dotnet_naming_rule.async_methods_should_be_pascalcase_async.symbols = async_methods dotnet_naming_rule.async_methods_should_be_pascalcase_async.style = pascalcase_async dotnet_naming_symbols.async_methods.applicable_kinds = method dotnet_naming_symbols.async_methods.applicable_accessibilities = * dotnet_naming_symbols.async_methods.required_modifiers = async dotnet_naming_style.pascalcase_async.required_suffix = Async dotnet_naming_style.pascalcase_async.capitalization = pascal_case Creating the naming convention for local variables and parameters 1 2 3 4 5 6 7 dotnet_naming_rule.locals_and_parameters_should_be_pascal_case.severity = error dotnet_naming_rule.locals_and_parameters_should_be_pascal_case.symbols = locals_and_parameters dotnet_naming_rule.locals_and_parameters_should_be_pascal_case.style = camel_case dotnet_naming_symbols.locals_and_parameters.applicable_kinds = parameter, local dotnet_naming_style.camel_case.capitalization = camel_case How to ignore the CA1707 rule (Identifiers should not contain underscores) on test projects Some conventions for naming test methods use underscore. If that is your case, you will receive a violation for the CA1707 rule. To disable the rule only on the test project, create a file named GlobalSuppressions.cs in the root of your test project with the content below.\n1 2 3 using System.Diagnostics.CodeAnalysis; [assembly: SuppressMessage(\u0026#34;Naming\u0026#34;, \u0026#34;CA1707:Identifiers should not contain underscores\u0026#34;, Justification = \u0026#34;Not applicable for test names\u0026#34;, Scope = \u0026#34;module\u0026#34;)] Third-party analyzers There are third-party analyzers that can have additional rules that can be useful. These are some:\nRoslynator StyleCop Sonar Analyzer References and Links https://learn.microsoft.com/en-us/visualstudio/code-quality/install-net-analyzers?view=vs-2022 https://www.jetbrains.com/help/rider/Using_EditorConfig.html#export-code-style-settings https://marketplace.visualstudio.com/items?itemName=MadsKristensen.EditorConfig ","date":"2022-10-25T08:00:00-03:00","dateFormatted":"25 Oct 2022","image":"https://genezini.com/blog/p/enforcing-.net-code-style-rules-at-compile-time/cover.png","permalink":"https://genezini.com/blog/p/enforcing-.net-code-style-rules-at-compile-time/","title":"Enforcing .NET code style rules at compile time"},{"content":"Introduction When working with other people and multiple editors/IDEs, it is common to have different editor settings, losing consistency in formatting styles of the code. For example:\nUsing tabs/spaces and different sizes of indentation, making your code harder to read; Using different encoding between files, causing hard to find bugs at runtime (showing invalid characters) and breaking automated tests. In this post I\u0026rsquo;ll show how to maintain a standard for everyone who works in the code, no matter the editor used, and in a next post I\u0026rsquo;ll show how to enforce these (and other) rules on build and in the continuous integration pipeline.\nEnter the EditorConfig file The EditorConfig file is used by editors and IDEs to define editor preferences for the project. Without it, IDEs and editors will use their general configuration, causing divergences in the files edited on them.\nMany editors and IDEs support the EditorConfig file by default, and others have plugins to support it. Here are some:\nVisual Studio (Built-in); JetBrains Rider (Built-in); GitHub (Built-in); VS Code (Plugin); Vim (Plugin); Emacs (Plugin); Sublime (Plugin); It has a default name of .editorconfig and is an INI file where sections are filename filters, for instance:\n[*.cs] for .cs files; [scripts/**.js] for javascript files inside the scripts folder and subdirectories; [{package.json}] for the package.json file only. More details here\nThe EditorConfig can be put in any directory and be overridden by EditorConfig files in child directories, but for better visibility, they should be in the same directory as the .NET solution file.\nAdding an EditorConfig to your project To include an EditorConfig to your project, just create a file named .editorconfig in the same directory of your solution file (.sln) with the content below.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # Top-most EditorConfig file. root = true # Section for C# files # All rules below apply only to .cs files [*.cs] #### Core EditorConfig Options #### # Indentation and spacing indent_style = space indent_size = 4 # New line preferences end_of_line = crlf insert_final_newline = false trim_trailing_whitespace = true # Charset preference charset = utf-8 This file set these rules:\nroot: No editorconfig file in parent directories will be read; indent_style: Indentation should use space; indent_size: Indentation should use 4 space characters; end_of_line: Lines should end with CR+LF characters; insert_final_newline: Do not automatically insert empty lines at the end of the files; trim_trailing_whitespace: Empty lines should have no space characters; charset: Files should be encoded with UTF-8 format. More details here.\n⚠️ GIT can change the line ends to LF on pushes to the repository and change it back to CRLF on checkouts. Then, the end_of_line settings can be safely set to CRLF. GIT for Windows suggests this configuration by default at installation. Details on how to configure are here. ⚠️ UTF-8 with BOM is not required nor recommended, according to the Unicode standard. More here. ℹ️ In .NET, the EditorConfig file can also be used to define analyzers rules and severities specific to the .NET environment. In a next post, I\u0026rsquo;ll show how to configure these rules. Appling the new formatting rules to code When we change the formatting rules for an existing project, the changes are not applied automatically. We need to manually trigger an auto-format.\nVisual Studio First, we have to include the Format document fixer to a Code Cleanup profile.\n1 - Navigate to Analyze \u0026gt; Code Cleanup \u0026gt; Configure Code Cleanup.\n2 - Include the Format document fixer for the profile you wish to run on save.\n3 - On the menu, select Analyze \u0026gt; Code Cleanup \u0026gt; Run Code Cleanup (Yout Profile) on Solution.\n⚠️ Visual Studio\u0026rsquo;s format document doesn\u0026rsquo;t change the file encoding. You have to use the dotnet-format or another tool for that. VS Code VS Code doesn\u0026rsquo;t have a feature to format all files at once. You have to use the Format Files extension.\n1 - Install the Format Files extension.\n2 - Navigate to View \u0026gt; Command Palette or press Ctrl+Shift+P.\n3 - Select the Start Format Files: Workspace or Start Format Files: From Glob option.\nJetBrains Rider Select Code \u0026gt; Reformat Code or press Ctrl+Alt+Enter.\nFormat on save When we create an EditorConfig file, the supported editors will also use the configurations for their auto-format features.\nHere I show how to enable the auto-format on file save in some editors.\nFormat on save in Visual Studio Visual Studio 2022 doesn\u0026rsquo;t have a format on save feature, but it can run a Code Cleanup on save. This way we can configure a Code Cleanup profile to run a Format document and use it on save.\nℹ️ For Visual Studio 2019, there is an extension that enables the same feature. 1 - Configure your Code Cleanup profile to run the Format document fixer, as shown in the previous section of this post.\n2 - Navigate to Tools \u0026gt; Options \u0026gt; Text Editor \u0026gt; Code Cleanup.\n3 - Check the Run Code Cleanup profile on Save option and select the Code Cleanup profile to run on save.\nNow Visual Studio will format your files on every save.\nFormat on save in VS Code 1 - Navigate to File \u0026gt; Preferences \u0026gt; Settings.\n2 - Navigate to Text Editor \u0026gt; Formatting or search for editor.formatOnSave.\n3 - Check the Editor: Format On Save option.\nFormat on save in JetBrains Rider 1 - Navigate to File \u0026gt; Settings \u0026gt; Tools \u0026gt; Actions on Save.\n2 - Check the Reformat and Cleanup Code option.\n3 - Select the Reformat Code profile.\nReferences and Links https://editorconfig.org/ https://devblogs.microsoft.com/visualstudio/bringing-code-cleanup-on-save-to-visual-studio-2022-17-1-preview-2/ https://learn.microsoft.com/en-us/visualstudio/ide/create-portable-custom-editor-options?view=vs-2022 https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-format ","date":"2022-10-18T08:00:00-03:00","dateFormatted":"18 Oct 2022","image":"https://genezini.com/blog/p/defining-formatting-rules-in-.net-with-editorconfig/cover.png","permalink":"https://genezini.com/blog/p/defining-formatting-rules-in-.net-with-editorconfig/","title":"Defining formatting rules in .NET with EditorConfig"},{"content":"Introduction Although there are many definitions about the scope of an integration test, Martin Fowler defines Narrow integration tests, where the integration with other systems are tested using mocks, and Broad integration tests, where they communicate using real APIs.\nIn this post, I\u0026rsquo;ll explain how to create mocks for HTTP APIs in narrow integration tests using the WireMock.Net library.\nWhat should we mock? Vladimir Khorikov has a concept of managed dependencies and unmanaged dependencies, which I consider complementary to Martin Fowler\u0026rsquo;s, to choose what should be to mocked.\nManaged dependencies are external systems controlled by us and accessed only by our application (for example, a database). On the other side, unmanaged dependencies are external systems not controlled by us or also accessed by other applications (like a third party API or a message broker).\nVladimir says that we should test our system against managed dependencies, while mocking unmanaged dependencies. I believe this definition is more like a guideline than a rule. For example, in a scenario where our application posts in a message broker for other system to read, that is, the message broker is an unmanaged dependency, we could test the integration with the message broker to validate that the message is being written in the right format (according to contract). This can have value if we want to test if updates to the library used to communicate with the message broker didn\u0026rsquo;t introduce breaking changes in the message.\nWhy use mocks? The reason we use integration tests is to test our components (or classes), which are tested independently in unit tests, working in communication with each other. When we interact with an API, we follow a protocol and trust a contract of communication, that is, that the API will accept parameters X as input and will return an response Y.\nThat way, the inner works of that external API is not in the scope of our integration tests.\nThis doesn\u0026rsquo;t remove the requirement of functional tests; it only reduces the amount of those tests, which are more expensive to execute.\nReducing the integration tests only to our application, we have some benefits:\nSpeed of the tests, because we remove the network latency; No need of data in external systems to execute the tests; Reduced brittleness of the tests, that could break in case of the external API instability or external data that changed; More trust in the test results. Using WireMock.Net In this example, I\u0026rsquo;ve built an API that consumes the PokéAPI service to look for a Pokémon data and return it to the client.\nController The controller is simple and use the Refit library to abstract the PokéAPI call and then, returns the data.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 using Microsoft.AspNetCore.Mvc; using Refit; namespace PokemonInfoAPI.Controllers { [ApiController] [Route(\u0026#34;[controller]\u0026#34;)] public class PokemonInfoController : ControllerBase { private readonly IConfiguration _configuration; public PokemonInfoController(IConfiguration configuration) { _configuration = configuration; } [HttpGet(\u0026#34;{pokemonName}\u0026#34;)] public async Task\u0026lt;ActionResult\u0026lt;PokemonInfo\u0026gt;\u0026gt; GetAsync(string pokemonName) { try { var pokeApi = RestService.For\u0026lt;IPokeApi\u0026gt;(_configuration[\u0026#34;PokeApiBaseUrl\u0026#34;]); return Ok(await pokeApi.GetPokemonInfo(pokemonName)); } catch (ApiException ex) { if (ex.StatusCode == System.Net.HttpStatusCode.NotFound) { return NotFound(); } return StatusCode(500); } } } } Default integration test We start with a default integration test, using ASP.NET Core\u0026rsquo;s WebApplicationFactory class. The test creates an instance of our application e makes a request to the /pokemoninfo endpoint with the parameter charmander. For now, our test will call the PokéAPI.\n💡 You can use any class of your API project to instanciate the WebApplicationFactory in your tests. If you\u0026rsquo;re using top-level statements in your application, you can use a controller class. For example, WebApplicationFactory\u0026lt;PokemonInfoController\u0026gt;. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 using FluentAssertions; using Microsoft.AspNetCore.Mvc.Testing; using System.Net; using System.Text.Json; namespace PokemonInfoAPI.IntegrationTests { public class PokemonInfoTests: IClassFixture\u0026lt;WebApplicationFactory\u0026lt;Program\u0026gt;\u0026gt; { private readonly WebApplicationFactory\u0026lt;Program\u0026gt; _factory; public PokemonInfoTests(WebApplicationFactory\u0026lt;Program\u0026gt; factory) { _factory = factory; } [Fact] public async Task Get_Existing_Pokemon_Returns_200() { //Arrange var HttpClient = Factory.CreateClient(); //Act var HttpResponse = await HttpClient.GetAsync(\u0026#34;/pokemoninfo/charmander\u0026#34;); //Assert HttpResponse.StatusCode.Should().Be(HttpStatusCode.OK); var ResponseJson = await HttpResponse.Content.ReadAsStringAsync(); var PokemonInfo = JsonSerializer.Deserialize\u0026lt;PokemonInfo\u0026gt;(ResponseJson); PokemonInfo.Should().BeEquivalentTo(ResponseObj); } } } Setting up a mock for PokéAPI WireMock.Net is a library that let you create mocks for HTTP APIs. It creates a web server in the same process of our test and exposes an URL to be used by out application during the tests.\nUsing WireMock.Net and WebApplicationFactory we will have this scenario:\nFirst, I install the WireMock.Net nuget package in my tests project.\nUsing Visual Studio Package Manager Install-Package WireMock.Net Or\nUsing .NET CLI dotnet add package WireMock.Net Starting WireMock.Net server To start the WireMock.Net server, I call the Start method of the WireMockServer class, and it returns an object with the server data.\n1 var WireMockSvr = WireMockServer.Start(); Overriding out application configurations With the server started, I override the PokeApiBaseUrl parameter, which holds the PokéAPI URL, in my application configurations using the method WithWebHostBuilder of the WebApplicationFactory:\n1 2 3 4 5 6 var HttpClient = _factory .WithWebHostBuilder(builder =\u0026gt; { builder.UseSetting(\u0026#34;PokeApiBaseUrl\u0026#34;, WireMockSvr.Url); }) .CreateClient(); Mocking the /pokemon endpoint Then, I create the mock for the /pokemon endpoint receiving the parameter value charmander.\nIn the example below, I\u0026rsquo;m using the AutoFixture library to generate an object with random values, that will be returned by the mocked API.\nℹ️ By using an object, I can compare the return of my application with this object, but it\u0026rsquo;s also possible to configure the return based on an file with a JSON, with the WithBodyFromFile method. Also, I set the headers that will be returned and the HTTP status of the response.\n1 2 3 4 5 6 7 8 9 10 11 12 13 Fixture fixture = new Fixture(); var ResponseObj = fixture.Create\u0026lt;PokemonInfo\u0026gt;(); var ResponseObjJson = JsonSerializer.Serialize(ResponseObj); WireMockSvr .Given(Request.Create() .WithPath(\u0026#34;/pokemon/charmander\u0026#34;) .UsingGet()) .RespondWith(Response.Create() .WithBody(ResponseObjJson) .WithHeader(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;) .WithStatusCode(HttpStatusCode.OK)); After that, my application inside the tests will be using the mocked version of the PokéAPI.\nComplete test code 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 [Fact] public async Task Get_Existing_Pokemon_Returns_200() { //Arrange var WireMockSvr = WireMockServer.Start(); var HttpClient = _factory .WithWebHostBuilder(builder =\u0026gt; { builder.UseSetting(\u0026#34;PokeApiBaseUrl\u0026#34;, WireMockSvr.Url); }) .CreateClient(); Fixture fixture = new Fixture(); var ResponseObj = fixture.Create\u0026lt;PokemonInfo\u0026gt;(); var ResponseObjJson = JsonSerializer.Serialize(ResponseObj); WireMockSvr .Given(Request.Create() .WithPath(\u0026#34;/pokemon/charmander\u0026#34;) .UsingGet()) .RespondWith(Response.Create() .WithBody(ResponseObjJson) .WithHeader(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;) .WithStatusCode(HttpStatusCode.OK)); //Act var HttpResponse = await HttpClient.GetAsync(\u0026#34;/pokemoninfo/charmander\u0026#34;); //Assert HttpResponse.StatusCode.Should().Be(HttpStatusCode.OK); var ResponseJson = await HttpResponse.Content.ReadAsStringAsync(); var PokemonInfo = JsonSerializer.Deserialize\u0026lt;PokemonInfo\u0026gt;(ResponseJson); PokemonInfo.Should().BeEquivalentTo(ResponseObj); WireMockSvr.Stop(); } Example of an unsuccessfull API call scenario Based on the contract of the API, we know that it return the status 404 (Not Found) when the parameter is not a valid Pokémon name, so I created a mock that returns this status for the parameter value woodywoodpecker and assert that my application response is correct for this scenario.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 [Fact] public async Task Get_NotExisting_Pokemon_Returns_404() { //Arrange var WireMockSvr = WireMockServer.Start(); var Factory = _factory.WithWebHostBuilder(builder =\u0026gt; { builder.UseSetting(\u0026#34;PokeApiBaseUrl\u0026#34;, WireMockSvr.Url); }); var HttpClient = Factory.CreateClient(); Fixture fixture = new Fixture(); WireMockSvr .Given(Request.Create() .WithPath(\u0026#34;/pokemon/woodywoodpecker\u0026#34;) .UsingGet()) .RespondWith(Response.Create() .WithHeader(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;) .WithStatusCode(HttpStatusCode.NotFound)); //Act var HttpResponse = await HttpClient .GetAsync(\u0026#34;/pokemoninfo/woodywoodpecker\u0026#34;); //Assert HttpResponse.StatusCode.Should().Be(HttpStatusCode.NotFound); WireMockSvr.Stop(); } Troubleshooting In this post I explain how to troubleshoot problems with WireMock.Net and talk about some common problems that I see in my daily work.\nSource code https://github.com/dgenezini/PokemonInfoAPIWireMockTests\nReferences and links https://martinfowler.com/bliki/IntegrationTest.html https://khorikov.org/posts/2021-11-29-unmanaged-dependencies-explained/ https://github.com/WireMock-Net/WireMock.Net https://github.com/reactiveui/refit https://github.com/AutoFixture/AutoFixture https://github.com/fluentassertions/fluentassertions ","date":"2022-09-25T08:00:00-03:00","dateFormatted":"25 Sep 2022","image":"https://genezini.com/blog/p/integration-tests-without-api-dependencies-with-asp.net-core-and-wiremock.net/cover.jpg","permalink":"https://genezini.com/blog/p/integration-tests-without-api-dependencies-with-asp.net-core-and-wiremock.net/","title":"Integration tests without API dependencies with ASP.NET Core and WireMock.Net"}]