<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en">
    <title>NIYONSHUTI Emmanuel</title>
    <subtitle>NIYONSHUTI Emmanuel Blog</subtitle>
    <link rel="self" type="application/atom+xml" href="https://emmanuelniyonshuti.github.io/atom.xml"/>
    <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io"/>
    <generator uri="https://www.getzola.org/">Zola</generator>
    <updated>2026-11-12T00:00:00+00:00</updated>
    <id>https://emmanuelniyonshuti.github.io/atom.xml</id>
    <entry xml:lang="en">
        <title>switching git branches with unstaged changes</title>
        <published>2026-11-12T00:00:00+00:00</published>
        <updated>2026-11-12T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/switching-git-branches-with-unstaged-changes/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/switching-git-branches-with-unstaged-changes/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/switching-git-branches-with-unstaged-changes/">&lt;p&gt;This probably should have been a TIL, but it&#39;s a short option on git checkout that
I have been using for few months and thought was worth taking a note of here!&lt;/p&gt;
&lt;p&gt;Do you ever play around with something on main or master branch and after get it working the way you want, and then want to move those changes to the other branch, only to realize you can&#39;t simply git checkout into that branch because you have unstaged changes?&lt;/p&gt;
&lt;p&gt;Is this familiar?:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;error:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; Your local changes to the following files would be overwritten by checkout:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;        project_name/some_file.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;Please&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; commit your changes or stash them before you switch branches.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;Aborting&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;What I normally do is stash my changes &lt;code&gt;git stash&lt;/code&gt;, switch branch and pop the stash &lt;code&gt;git stash pop&lt;/code&gt;.
Often, I don&#39;t have to look up a particular stash with git stash list, because git stash pop attempts to apply my most recent stash and removes it from the stash list.&lt;/p&gt;
&lt;p&gt;While this has worked for me for a while, I&#39;ve found that using this option is cleaner than doing the whole stashing dance:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;git&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; checkout -m branch-name&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You can use this with git switch as well. &lt;code&gt;-m&lt;/code&gt; is short for &lt;code&gt;--merge&lt;/code&gt;, and it carries my local unstaged changes over to the branch I&#39;m checking out.&lt;/p&gt;
&lt;p&gt;bye!&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>Exploring Tera</title>
        <published>2026-11-11T00:00:00+00:00</published>
        <updated>2026-11-11T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/exploring-tera/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/exploring-tera/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/exploring-tera/">&lt;p&gt;My web blog &lt;a rel=&quot;external&quot; href=&quot;https://github.com/EmmanuelNiyonshuti/blog&quot;&gt;&lt;em&gt;&lt;u&gt;used to be&lt;/u&gt;&lt;/em&gt;&lt;/a&gt; and now I decided to
run it on zola, it&#39;s templating engine(Tera) seems the same as Jinja so far!.&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>lazy imports python3.15</title>
        <published>2026-08-01T00:00:00+00:00</published>
        <updated>2026-08-01T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/lazy-imports-python315/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/lazy-imports-python315/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/lazy-imports-python315/">&lt;h3 id=&quot;eager-imports&quot;&gt;eager imports&lt;/h3&gt;
&lt;p&gt;Python loads and executes a module the moment you import it.&lt;/p&gt;
&lt;p&gt;Let&#39;s take this toy example, let&#39;s say I have a module named &lt;code&gt;heavy.py&lt;/code&gt; pretending it&#39;s something like &lt;code&gt;matplotlib&lt;/code&gt; that I want to import and use in my code.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;#heavy.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;import&lt;/span&gt;&lt;span&gt; time&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;heavy.py: top-level code running&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;time&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;sleep&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;0.3&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;   # pretend this is the module importing its own dependencies and running its startup code&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;heavy.py: done&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; compute&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    return&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;some heavy computation result&amp;quot;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s import this in the python repl:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;$ python3&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Python&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 3.14&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;2&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt; (&lt;/span&gt;&lt;span&gt;main&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span&gt; Jan&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 14 2026&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 19&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;38&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;07&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;) [&lt;/span&gt;&lt;span&gt;Clang&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 21.1&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;4&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt; ]&lt;/span&gt;&lt;span&gt; on linux&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Type&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;help&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;copyright&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;credits&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; or&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;license&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; for&lt;/span&gt;&lt;span&gt; more information&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; heavy&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;heavy&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;py&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span&gt; top&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;-&lt;/span&gt;&lt;span&gt;level code running&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;heavy&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;py&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span&gt; done&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span&gt; heavy&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;compute&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;#39;some heavy computation result&amp;#39;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We can see that default python import system, loads and executes the top-level code of the module you import. Taking an example we used of  &lt;code&gt;matplotlib&lt;/code&gt; but in reality rather than using the toy script above, importing this package will pull in other packages like numpy, pillow, and more. Just like many other python packages, sometimes they depend on each other, and with how the import system in python works, this can add a small or significant overhead to your code depending on what you import and how you are importing them.&lt;/p&gt;
&lt;p&gt;Let&#39;s actually install matplotlib as the example to see how long the modules it imports and itself takes to run. Luckily, Python has a built-in way to see this kind of thing.
We can use &lt;code&gt;-X importtime&lt;/code&gt; flag and print, for every module imported, how long that module&#39;s own code took to run, and how long it took including everything it pulls in(in other words its dependencies).&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;$&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; python3 -X importtime -c &amp;quot;import matplotlib&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; 2&amp;gt;&amp;amp;1 |&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; grep&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; -E &amp;quot;numpy$|PIL$|matplotlib$&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;import&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; time: self&lt;/span&gt;&lt;span&gt; [us] &lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;|&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; cumulative&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; |&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; imported&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; package&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;import&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; time:&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;       957&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; |&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;      52165&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; |&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;     numpy&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;import&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; time:&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;       233&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; |&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;        526&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; |&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;       PIL&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;import&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; time:&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;      6169&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; |&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;     175962&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; |&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; matplotlib&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;On my computer, importing matplotlib alone costs about 176ms cumulative, but matplotlib&#39;s own code only accounts for 6ms of that (self). Most of the cost comes from everything it depends on. Numpy alone is 52ms cumulative, for example. Obviously this can vary depending on the computer, but there&#39;s still some kind of overhead because of how python&#39;s import system works. When your code imports many modules like this, which is normally the case, it can add up and contribute to your application&#39;s slow startup. You can check out the actual benchmarks in PEP 810, by the way -- I put the link in the conclusion section below.&lt;/p&gt;
&lt;h3 id=&quot;lazy-imports&quot;&gt;lazy imports&lt;/h3&gt;
&lt;p&gt;python3.15 comes with a feature called &lt;code&gt;lazy imports&lt;/code&gt;. The idea is that instead of loading a module the moment it&#39;s imported, the import is deferred and the module is only loaded when it&#39;s actually used. Think of a trick people already do by importing a module inside a function instead of at the top of the file, so it&#39;s only loaded when that function runs. it&#39;s basically the same idea, now built into the language.&lt;/p&gt;
&lt;p&gt;Let&#39;s try the same example, but now with python3.15&#39;s lazy import:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;$ python3&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;15&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Python&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 3.15&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;0b3&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt; (&lt;/span&gt;&lt;span&gt;main&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span&gt; Jun&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 23 2026&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 15&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;19&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;17&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;) [&lt;/span&gt;&lt;span&gt;Clang&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 22.1&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;3&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt; ]&lt;/span&gt;&lt;span&gt; on linux&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Type&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;help&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;copyright&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;credits&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; or&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;license&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; for&lt;/span&gt;&lt;span&gt; more information&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span&gt; lazy&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; heavy&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span&gt; heavy&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;compute&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;heavy&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;py&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span&gt; top&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;-&lt;/span&gt;&lt;span&gt;level code running&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;heavy&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;py&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span&gt; done&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;#39;heavy computation result&amp;#39;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As you can see, we use the &lt;code&gt;lazy&lt;/code&gt; keyword, and the module is only loaded the first time it&#39;s actually used. modules imported without the &lt;code&gt;lazy&lt;/code&gt; keyword are still imported eagerly, same as before. there&#39;s also a second way to opt in: a &lt;code&gt;__lazy_modules__&lt;/code&gt; list at the top of a module.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;__lazy_modules__&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt; [&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;#39;json&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;]&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;import&lt;/span&gt;&lt;span&gt; json&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;import&lt;/span&gt;&lt;span&gt; sys&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;#39;json&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; in&lt;/span&gt;&lt;span&gt; sys&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;modules&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;result&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span&gt; json&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;dumps&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;({&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;#39;name&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;#39;john&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;})&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;#39;json&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; in&lt;/span&gt;&lt;span&gt; sys&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;modules&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;$ python3&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;15&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; demo&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;False&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;True&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We can see that, the actual module loading is deferred until it&#39;s actually used.&lt;/p&gt;
&lt;h3 id=&quot;a-couple-of-gotchas&quot;&gt;a couple of gotchas&lt;/h3&gt;
&lt;p&gt;&lt;em&gt;you can lazy import something that doesn&#39;t exist&lt;/em&gt;&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;$ python3&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;15&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span&gt; lazy&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; from&lt;/span&gt;&lt;span&gt; json&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; idontexist&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This might look like a problem, I think it is easy to make this mistake. But I think it&#39;s exactly what you&#39;d expect from deferring the lookup and loading until first use. I think that&#39;s the point, right?&lt;/p&gt;
&lt;p&gt;&lt;em&gt;what if you use the module right at the top of your module?&lt;/em&gt;&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;lazy&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; yaml&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;import&lt;/span&gt;&lt;span&gt; sys&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;import&lt;/span&gt;&lt;span&gt; functools&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;loads&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span&gt; functools&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;partial&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;yaml&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;load&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; Loader&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span&gt;yaml&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;CSafeLoader&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;#39;yaml&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; in&lt;/span&gt;&lt;span&gt; sys&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;modules&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; demo&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    ...&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;AttributeError&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span&gt; module&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;#39;yaml&amp;#39;&lt;/span&gt;&lt;span&gt; has no attribute&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;#39;CSafeLoader&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt; Did you mean&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;#39;.SafeLoader&amp;#39;&lt;/span&gt;&lt;span&gt; instead of&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;#39;.CSafeLoader&amp;#39;&lt;/span&gt;&lt;span&gt;?&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;run it and you get an &lt;code&gt;AttributeError&lt;/code&gt;, because the module ends up loading right away. we&#39;re using it immediately at the top, so &lt;code&gt;lazy&lt;/code&gt; doesn&#39;t buy us anything here. I came across this library pretty recently which this example is copied from, anthony&#39;s &lt;a rel=&quot;external&quot; href=&quot;https://github.com/asottile/lazy-static&quot;&gt;lazy-static&lt;/a&gt;. I think you can obviously write the same check yourself, but it&#39;s worth knowing a small library already exists for it.&lt;/p&gt;
&lt;h3 id=&quot;conclusion&quot;&gt;conclusion&lt;/h3&gt;
&lt;p&gt;You can read more about lazy imports in &lt;a rel=&quot;external&quot; href=&quot;https://peps.python.org/pep-0810/&quot;&gt;PEP 810&lt;/a&gt;. Ultimately, I hope you got something out of this, and you can now go start experimenting with and adopting this new feature.&lt;/p&gt;
&lt;p&gt;bye!&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>cancelling Asyncio.TaskGroup without boilerplate</title>
        <published>2026-07-05T00:00:00+00:00</published>
        <updated>2026-07-05T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/cancelling-asyncio-taskgroup-without-boilerplate/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/cancelling-asyncio-taskgroup-without-boilerplate/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/cancelling-asyncio-taskgroup-without-boilerplate/">&lt;p&gt;Python3.15 is right around the corner as of when I am writing this article, its scheduled for its official stable release in october as always. I am running on its beta release &lt;code&gt;Python3.15.0b3&lt;/code&gt; and out of everything on the release notes, lazy imports, utf8 as default encoding, sentinel builtin, frozendict builtin type, the one I actually want to check on in this blog post today is &lt;code&gt;asyncio.TaskGroup.cancel&lt;/code&gt;. so this is that.&lt;/p&gt;
&lt;h3 id=&quot;cancelling-asyncio-taskgroup&quot;&gt;cancelling Asyncio.TaskGroup&lt;/h3&gt;
&lt;p&gt;There is a blog I have which was a fairly minimal walk through, or introduction, to asynchronous programming in python using AnyIO&#39;s &lt;code&gt;create_task_group&lt;/code&gt; for spawning different tasks. Task Groups, or nurseries in Trio lingo, have their own Cancel Scopes, cancelling them is as easy as calling &lt;code&gt;tg.cancel_scope.cancel()&lt;/code&gt;. Cancel scopes are not a thing in Python AsyncIO, and part of why is that the cancellation semantics of Trio and asyncio differ, the former uses Level cancellation and the latter uses edge cancellation. you can use Cancel Scopes when you run AnyIO, because of what AnyIO does around its asyncio backend, but asyncio itself doesn&#39;t have this.&lt;/p&gt;
&lt;p&gt;If you wanted to short circuit a &lt;code&gt;TaskGroup&lt;/code&gt; early, you had to reach for the task the group is running under and cancel that, this leads to some boilerplate code where you have to catch the &lt;code&gt;CancelledError&lt;/code&gt; that will propagate outside the TaskGroup. this is not very safe, cancelling it cancels the whole task, not just whatever&#39;s inside your taskgroup, it is possible you can end up taking down code that has nothing to do with the group.&lt;/p&gt;
&lt;h3 id=&quot;naming-things-can-be-hard-sometimes&quot;&gt;naming things can be hard sometimes!&lt;/h3&gt;
&lt;p&gt;It was actually almost named &lt;code&gt;TaskGroup.stop&lt;/code&gt;. Guido seemed to lean towards calling this API &lt;code&gt;TaskGroup.stop&lt;/code&gt;. his premise being that calling it &lt;code&gt;TaskGroup.cancel&lt;/code&gt; implies that it will raise a &lt;code&gt;CancelledError&lt;/code&gt; that&#39;s expected to propagate out of the taskgroup. but it actually doesn&#39;t propagate, it gets caught, and the taskgroup exits clean. eventually though, Guido came around to naming it TaskGroup.cancel, since others preferred that name, and Yury, one of the python core devs, also thought cancel was the more fitting name.&lt;/p&gt;
&lt;h3 id=&quot;what-it-entails&quot;&gt;What it entails&lt;/h3&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# code copied from cpython3 https://github.com/python/cpython/issues/108951#issuecomment-1707742153&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;import&lt;/span&gt;&lt;span&gt; asyncio&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; t1&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    async with&lt;/span&gt;&lt;span&gt; asyncio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;TaskGroup&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; as&lt;/span&gt;&lt;span&gt; tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;create_task&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;asyncio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;sleep&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;1&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;))&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;_parent_task&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;cancel&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt; # asyncio.current_task().cancel()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    assert&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; False&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;not reached&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;try&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    asyncio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;t1&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;())&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;    # Cancelled&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    assert&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; False&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;not reached&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;except&lt;/span&gt;&lt;span&gt; asyncio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;CancelledError&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;    print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;Cancelled&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Updated to use python&amp;gt;=315 &lt;code&gt;TaskGroup.cancel&lt;/code&gt;&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;import&lt;/span&gt;&lt;span&gt; asyncio&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; t1&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    async with&lt;/span&gt;&lt;span&gt; asyncio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;TaskGroup&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; as&lt;/span&gt;&lt;span&gt; tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;create_task&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;asyncio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;sleep&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;1&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;))&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;cancel&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;    print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;exited clean, no exception to catch&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;asyncio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;t1&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;())&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt; # exited clean, no exception to catch&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;h3 id=&quot;why-care&quot;&gt;why care&lt;/h3&gt;
&lt;p&gt;If you&#39;ve never needed to bail out of a &lt;code&gt;TaskGroup&lt;/code&gt; early, this feature will most likely not be something you care about. its a small api and I think it removes a real wart. worth knowing it exists next time a task group of yours needs to know when to stop.&lt;/p&gt;
&lt;p&gt;bye!&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>cat spelled backwards</title>
        <published>2026-05-28T00:00:00+00:00</published>
        <updated>2026-05-28T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/til/cat-spelled-backwards/"/>
        <id>https://emmanuelniyonshuti.github.io/til/cat-spelled-backwards/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/til/cat-spelled-backwards/">&lt;p&gt;we can use &lt;code&gt;tac&lt;/code&gt; to print a file with its lines in reverse order.&lt;/p&gt;
&lt;p&gt;e.g: for log files which we care about the most recent entries,
we can use &lt;code&gt;tac&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;print the whole log file in reverse order.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;tac tmp/app.log&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;print 5 most recent log entries.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;tac tmp/app.log | head -5&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;</content>
        
    </entry>
    <entry xml:lang="en">
        <title>pytest fixture scopes explained through scope mismatch</title>
        <published>2026-05-27T00:00:00+00:00</published>
        <updated>2026-05-27T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/downward-fixture-dependency-will-mess-up-your-teardown/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/downward-fixture-dependency-will-mess-up-your-teardown/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/downward-fixture-dependency-will-mess-up-your-teardown/">&lt;p&gt;don&#39;t depend downward on pytest fixtures. what do I mean by that?&lt;/p&gt;
&lt;p&gt;if you test your code (which I assume you actually do!) you&#39;ll find yourself needing
to provide some data or establish some state before your tests run — and to do that,
you create fixtures and request them in your tests. this is what they call the setup phase,
and after your tests run, your fixtures get torn down in order of their scope.&lt;/p&gt;
&lt;br /&gt;
let&#39;s first remind ourselves that fixtures in pytest have a scope hierarch: &lt;br /&gt;
&lt;p&gt;&lt;code&gt;function&lt;/code&gt; &amp;lt; &lt;code&gt;class&lt;/code&gt; &amp;lt; &lt;code&gt;module&lt;/code&gt; &amp;lt; &lt;code&gt;package&lt;/code&gt; &amp;lt; &lt;code&gt;session&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;you can not for example have a module scoped fixture depending on a function scoped fixture. a fixture should depend on either its same scoped fixture or the higher scoped fixture above it.
sometimes, you might be tempted to do something close to this:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;@pytest&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;fixture&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt; # function scoped by default&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;db&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    user&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span&gt; db&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;create&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;email&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;tmp@test.com&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; role&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;user&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    yield&lt;/span&gt;&lt;span&gt; user&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;delete&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;@pytest&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;fixture&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;scope&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;session&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; admin_client&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    return&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; AdminClient&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;base_url&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;http://localhost&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; acting_as&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span&gt;user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;id&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;you want an &lt;code&gt;admin_client&lt;/code&gt; across the whole test session(&lt;span className=&quot;font-medium&quot;&gt; may be its expensive to spin up!&lt;/span&gt;) and you are tempted to think that, since you already have a &lt;code&gt;user&lt;/code&gt; fixture why not hand it to this &lt;code&gt;admin_client&lt;/code&gt; fixture. this will just mess up the teardown phase of pytest fixtures.&lt;br /&gt; &lt;code&gt;user&lt;/code&gt; fixture will be torn down on every test function that request it but &lt;code&gt;admin_client&lt;/code&gt; fixture is going to stay for the whole test session hence pytest will complain about fixture scope mismatch. instead, if a fixture depends on say a &lt;code&gt;function&lt;/code&gt;-scoped fixture, it should itself be &lt;code&gt;function&lt;/code&gt;-scoped and this applies going up too.&lt;br /&gt; &lt;br /&gt;
in the above code, I would rather make a separate session scoped user fixture and leave function scoped user fixture for tests that need per-test isolation.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;@pytest&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;fixture&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;scope&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;session&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; session_user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;db&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    user&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span&gt; db&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;create&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;email&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;tmp@test.com&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; role&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;user&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    yield&lt;/span&gt;&lt;span&gt; user&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;delete&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;@pytest&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;fixture&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;scope&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;session&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; admin_client&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;session_user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    return&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; AdminClient&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;base_url&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;http://localhost&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; acting_as&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span&gt;session_user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;id&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;but then what if the user admin_client needs have to be fresh for each test (may be they have to have different roles, etc)? well, then admin_client should be a function scoped as well. we would drop it down to function scoped and bear whatever the setup cost it takes.&lt;/p&gt;
&lt;br /&gt;
ultimately, I think this hierarchy dependency makes sense and is kinda obvious too. I mean, would we have a module depending on an internal state of one of its callers? we can think of it in this way: &lt;br /&gt; *the more tests a fixture is shared across, the less it should know about any individual test and depending downward messes that up.*
&lt;p&gt;&lt;span className=&quot;font-semibold rounded-sm&quot;&gt;don&#39;t try to saw off the branch you&#39;re sitting on.&lt;/span&gt; bye!&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>blogging and llms</title>
        <published>2026-05-21T00:00:00+00:00</published>
        <updated>2026-05-21T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/blogging-and-llms/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/blogging-and-llms/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/blogging-and-llms/">&lt;p&gt;I don&#39;t really like some of my past blog posts, even though they&#39;re helpful and I sometimes revisit them to refresh on something.
Early on, the final draft of almost every post I wrote would go through an LLM before publishing.
After writting, i&#39;d prompt an LLM like this:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;hey, I am writting a technical blog about x_topic on my personal blogging website and english is not my first language.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;I need you to go through this draft I have written , find grammatical issues or other issues worth refining or improving and give me back a&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;cleaned up blog post I can publish. PLEASE DON&amp;#39;T CHANGE THE TONE AND SCOPE OF THIS DRAFT.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Here is the blog post:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;[paste blog contents]&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;then I would scan the spitted out draft, sometimes I would find it trying so hard to cover every single detail that I didn&#39;t want to have.
then skim through it and remove some things or start reconsidering leaving them that way. sometimes would even take more time to review the draft than time taken to write the first draft.
I should have improved the prompt? I did try that a couple of times, but I never really got past that example.
It mostly turns into going back and forth with an LLM and the time spent going back and forth with the LLM just doesn&#39;t match the value of the output I get, or the output I actually care about.&lt;/p&gt;
&lt;p&gt;That’s like the downside of it, but there’s also a bit of a greener grass on the other side. Sometimes it just points out that something feels off, which makes me go and look it up to confirm.
I don’t think it’s necessarily bad to be nudged by a machine when something isn’t quite right in your writting.
What I don’t like is the overconfidence and hallucinations that sometimes come with this.
When you talk about something fairly new, it can confidently push corrections even when it’s wrong.&lt;/p&gt;
&lt;p&gt;Ultimately, I&#39;m not going to hand over my original writing to an LLM for polishing the same way anymore.
And I know the debate about using AI or not even for this kind of thing is a long old one.
At this point I can&#39;t imagine not using some of these tools. Now, Instead of feeding my entire blog post into it, I&#39;d rather just ask about a specific sentence or paragraph when I feel stuck finding the right way to express something.&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>dynamic access to async fixture value within async test</title>
        <published>2026-04-26T00:00:00+00:00</published>
        <updated>2026-04-26T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/til/dynamic-access-to-async-fixture-value-within-async-test/"/>
        <id>https://emmanuelniyonshuti.github.io/til/dynamic-access-to-async-fixture-value-within-async-test/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/til/dynamic-access-to-async-fixture-value-within-async-test/">&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;import&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; pytest&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;pytestmark&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; = pytest.mark.anyio&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;@pytest.fixture&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;def&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; anyio_backend&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    return&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;asyncio&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;@pytest.fixture&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;async&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; def foo&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    return&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;foo&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;async&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; def test_something&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;request,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; foo&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    assert&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; request.getfixturevalue&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;&amp;#39;foo&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; == &amp;quot;foo&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As it turns out, this would not be possible if we didn&#39;t declare the &lt;code&gt;foo&lt;/code&gt; fixture parameter inside our async test. By declaring it, anyio will run foo during test setup in &lt;code&gt;run_until_complete&lt;/code&gt;, and later on when we try to access it in the test, it will be retrieved from the cache. Otherwise (if we didn&#39;t explicitly declare it for some reason), it will try to re-enter an already running event loop and we will hit a &lt;code&gt;RuntimeError&lt;/code&gt;. this specific case is not yet supported in either anyio or pytest-asyncio.&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>Concurrent Tasks with AnyIO Task Groups</title>
        <published>2026-04-21T00:00:00+00:00</published>
        <updated>2026-04-21T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/doing-many-things-at-the-same-time-with-anyio-create-task-group/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/doing-many-things-at-the-same-time-with-anyio-create-task-group/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/doing-many-things-at-the-same-time-with-anyio-create-task-group/">&lt;p&gt;we can use AnyIO &lt;code&gt;create_task_group&lt;/code&gt; to run tasks concurrently on a specific backend that we chose. I am sure you have written or at least came across python code with &lt;code&gt;async def&lt;/code&gt; or even &lt;code&gt;async with&lt;/code&gt; and &lt;code&gt;await&lt;/code&gt; in their syntax. these are coroutine functions and async context managers. in coming sentences I might use abbr like &lt;code&gt;coro&lt;/code&gt; to refer to coroutines.&lt;/p&gt;
&lt;h3 id=&quot;coroutines-and-generators&quot;&gt;coroutines and generators&lt;/h3&gt;
&lt;p&gt;a coroutine object is able to suspend its execution and remembers where it left off.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; func&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;...&lt;/span&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;     print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;hello&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;...&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; func&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span&gt;coroutine&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;font-style: italic;&quot;&gt; object&lt;/span&gt;&lt;span&gt; func at&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; 0x&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;72d0251365c0&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;not that different from a generator, in a sense that it is also able to pause and resume from where it left.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; fn&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;...&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    yield&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 1&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;...&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    yield&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 2&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;...&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; fn&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span&gt;generator&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;font-style: italic;&quot;&gt; object&lt;/span&gt;&lt;span&gt; fn at&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; 0x&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;72d024ee14e0&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;calling a coro function only creates a coro object, it doesn&#39;t run it like a normal function would. to run it, it needs an event loop. with the generator above I
could loop through its values or call &lt;code&gt;next()&lt;/code&gt; on it until it&#39;s exhausted and hits
&lt;code&gt;StopIteration&lt;/code&gt;. I could not do the same with a coro object. instead, only the event
loop knows how to do the pausing and resuming with that coro. the event loop
recognizes a coroutine function, and whenever it hits an &lt;code&gt;await&lt;/code&gt; inside it, it
suspends execution there and only comes back to it once whatever was being awaited
has completed.&lt;/p&gt;
&lt;h3 id=&quot;task-groups-and-anyio&quot;&gt;task groups and AnyIO&lt;/h3&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; func_1&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;...&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;     await&lt;/span&gt;&lt;span&gt; anyio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;sleep&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;1&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;...&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; main&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;...&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;     async with&lt;/span&gt;&lt;span&gt; anyio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;create_task_group&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; as&lt;/span&gt;&lt;span&gt; tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;...         tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;start_soon&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;func_1&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;...&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span&gt; anyio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;main&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; backend&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;trio&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;we established that a coro needs an event loop to actually run. &lt;code&gt;anyio.run&lt;/code&gt; is how
you hand your entry coroutine to one. it spins up the event loop, runs your coroutine
to completion, then tears it down. it&#39;s the same as calling &lt;code&gt;asyncio.run&lt;/code&gt; or &lt;code&gt;trio.run&lt;/code&gt;
directly depending on the backend. in a web framework, you won&#39;t call this yourself, the framework does it under the hood.&lt;/p&gt;
&lt;p&gt;we could run many things inside the &lt;code&gt;anyio.create_task_group()&lt;/code&gt; context manager and it will
only close after all tasks have completed. notice the &lt;code&gt;tg.start_soon&lt;/code&gt;, that won&#39;t run
inline right where we called it instead it schedules it for execution on the next event
loop iteration.that&#39;s the default behavior.
these tasks will run on &lt;code&gt;trio&lt;/code&gt; since that&#39;s what&#39;s specified as the backend, by default
AnyIO picks &lt;code&gt;asyncio&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;you might wonder why not just use one of these async libraries directly. asyncio has some well known &lt;a rel=&quot;external&quot; href=&quot;https://sailor.li/asyncio#major-2&quot;&gt;shortcomings&lt;/a&gt; and Trio was actually
designed from the ground up to fix them, stricter task and cancellation handling being
the core of it, though that&#39;s a rabbit hole of its own. the problem is you can&#39;t just drop
Trio into an asyncio codebase or the other way around. AnyIO bring those
better ideas to asyncio as well, now we can write code once and choose which backend runs it.&lt;/p&gt;
&lt;p&gt;taking a simple small example below, say you have an API that requires authentication before any endpoint is reachable. you cannot fire requests until the token is live. you can use &lt;code&gt;tg.start()&lt;/code&gt; to hold the caller right there until the task signals it&#39;s ready via &lt;code&gt;task_status.started()&lt;/code&gt;. once auth confirms itself, three requests go out concurrently, and &lt;code&gt;tg.create_task()&lt;/code&gt; hands back a &lt;code&gt;TaskHandle&lt;/code&gt; for each. this is something asyncio&#39;s &lt;code&gt;TaskGroup&lt;/code&gt; has no concept of. after the group closes you can read the results right off the handles.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;import&lt;/span&gt;&lt;span&gt; anyio&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; anyio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;abc&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; TaskStatus&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; authenticate&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;task_status&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;&quot;&gt; TaskStatus&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;&quot;&gt; anyio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;&quot;&gt;TASK_STATUS_IGNORED&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;) -&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; None&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    await&lt;/span&gt;&lt;span&gt; anyio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;sleep&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;0.3&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # token exchange&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    task_status&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;started&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;   # token is live, caller unblocks&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; fetch&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;endpoint&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;font-style: italic;&quot;&gt; str&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; delay&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;font-style: italic;&quot;&gt; float&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;) -&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;font-style: italic;&quot;&gt; str&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    await&lt;/span&gt;&lt;span&gt; anyio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;sleep&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;delay&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # simulates actual network time&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    return&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;font-style: italic;&quot;&gt; f&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #F5C2E7;&quot;&gt;{&lt;/span&gt;&lt;span&gt;endpoint&lt;/span&gt;&lt;span style=&quot;color: #F5C2E7;&quot;&gt;}&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;: 200 OK&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; main&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    async with&lt;/span&gt;&lt;span&gt; anyio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;create_task_group&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; as&lt;/span&gt;&lt;span&gt; tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;        await&lt;/span&gt;&lt;span&gt; tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;start&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;authenticate&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        handles&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt; [&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;            tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;create_task&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;fetch&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;/users&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 1.0&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)),&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;            tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;create_task&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;fetch&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;/orders&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 0.5&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)),&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;            tg&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;create_task&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;fetch&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;/products&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 1.5&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)),&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;        ]&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    for&lt;/span&gt;&lt;span&gt; h&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; in&lt;/span&gt;&lt;span&gt; handles&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;        print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;h&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;return_value&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;anyio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;main&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;h3 id=&quot;wrapping-up&quot;&gt;wrapping up&lt;/h3&gt;
&lt;p&gt;AnyIO consolidates and enhances both &lt;code&gt;trio&lt;/code&gt; and &lt;code&gt;asyncio&lt;/code&gt; and lets you select which
one to run your async code on. there&#39;s a good chance you might already be running
AnyIO in your code without noticing. for example, if you use FastAPI or Starlette, you&#39;re running
on AnyIO. what this article went through is really just the surface. underneath all
of this lives structured concurrency, cancellation scopes, and event loop mechanics
that go much deep a rabbit hole I&#39;m still trying to travel through myself. bye!&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>generic TypeVar</title>
        <published>2026-04-15T00:00:00+00:00</published>
        <updated>2026-04-15T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/til/generic-typevar/"/>
        <id>https://emmanuelniyonshuti.github.io/til/generic-typevar/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/til/generic-typevar/">&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;import random&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from typing import TypeVar&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;T = TypeVar(&amp;quot;T&amp;quot;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;def get_random(items: list[T]) -&amp;gt; T:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    return random.choice(items)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;using generic &lt;code&gt;TypeVar&lt;/code&gt; for type annotationing python code, here we expect a list of Generics (can be anything) but we are ensuring that there is some connection between the input and the output and our editor will be aware of that.&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>function with some of its argument values frozen</title>
        <published>2026-04-08T00:00:00+00:00</published>
        <updated>2026-04-08T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/til/new-version-of-a-function-with-some-of-its-arguments-values-frozen/"/>
        <id>https://emmanuelniyonshuti.github.io/til/new-version-of-a-function-with-some-of-its-arguments-values-frozen/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/til/new-version-of-a-function-with-some-of-its-arguments-values-frozen/">&lt;p&gt;consider this coroutine function:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;async def run_in_threadpool(func: Callable[P, T], *args: P.args, **kwargs: P.kwargs) -&amp;gt; T:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    func = functools.partial(func, *args, **kwargs)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    return await anyio.to_thread.run_sync(func)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;I actually got this coroutine function from &lt;a rel=&quot;external&quot; href=&quot;https://github.com/Kludex/starlette/blob/main/starlette/concurrency.py&quot;&gt;starlette&lt;/a&gt; and based on my limited knowledge of asgi frameworks, this is a coroutine function that will run a sync callable &lt;code&gt;func&lt;/code&gt; in a thread pool so it doesn&#39;t block or hog the event loop. in other words, if you for example create a sync endpoint, starlette does not have any problem handling it. it detects that the endpoint is not an &lt;code&gt;async callable&lt;/code&gt; and then run it in a thread pool instead.&lt;/p&gt;
&lt;p&gt;and what role doth&lt;code&gt;functools.partial&lt;/code&gt; serve in this affair?&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;func&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span&gt; functools&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;partial&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;func&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; *&lt;/span&gt;&lt;span&gt;args&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; **&lt;/span&gt;&lt;span&gt;kwargs&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span&gt;`&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;this is a higher order function, in my &lt;a rel=&quot;external&quot; href=&quot;https://emmanuelniyonshuti.github.io/blog/loading-and-dealing-with-environment-configurations-using-pydantic-settings&quot;&gt;environment configurations with pydantic settings&lt;/a&gt; blog, I used &lt;code&gt;@lru_cache()&lt;/code&gt; to cache loaded env configs so i don’t keep reading from the env file every time. so these higher-order functions either take functions as input or return new ones.&lt;/p&gt;
&lt;p&gt;what &lt;code&gt;partial&lt;/code&gt; is doing here is creating a new callable, a &lt;a rel=&quot;external&quot; href=&quot;https://docs.python.org/3/library/functools.html#partial-objects&quot;&gt;partial object&lt;/a&gt; where the arguments we passed are already attached to it. when starlette calls it in &lt;code&gt;run_sync&lt;/code&gt;, it doesn’t need to pass anything again, it just runs the function with those saved arguments.&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>PyPI Trusted Publishing with OpenID Connect and GitHub Actions</title>
        <published>2026-04-05T00:00:00+00:00</published>
        <updated>2026-04-05T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/til/pypi-trusted-publishing-with-openid-connect-and-github-actions/"/>
        <id>https://emmanuelniyonshuti.github.io/til/pypi-trusted-publishing-with-openid-connect-and-github-actions/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/til/pypi-trusted-publishing-with-openid-connect-and-github-actions/">&lt;p&gt;When I published my &lt;a rel=&quot;external&quot; href=&quot;https://pypi.org/project/whatdeps/&quot;&gt;first package&lt;/a&gt; on PyPi I used twine and manually entered my API key each time. Later I learned that you can publish automatically by pushing a version tag to GitHub and letting a GitHub Action handle authentication and upload with no stored credentials, thanks to PyPI&#39;s OpenID Connect trusted publishing.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;yaml&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; Publish&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;on&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;  push&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    tags&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;v*.*.*&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;jobs&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;  test&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    runs-on&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; ubuntu-latest&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    strategy&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;      matrix&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;        python-version&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt; [&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;#39;3.10&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;#39;3.11&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;#39;3.12&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;#39;3.13&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;#39;3.14&amp;#39;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;]&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    steps&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; uses&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; actions/checkout@v4&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; Install uv&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;        uses&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; astral-sh/setup-uv@v4&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; Install tox&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;        run&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; uv tool install tox --with tox-uv&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; Install Python&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;        run&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; uv python install ${{ matrix.python-version }}&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; Run tests&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;        run&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; tox -e ${{ matrix.python-version }}&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;  publish&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    needs&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; test&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    runs-on&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; ubuntu-latest&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    environment&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; pypi&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    permissions&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;      id-token&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; write&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    steps&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; uses&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; actions/checkout@v4&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; Install uv&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;        uses&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; astral-sh/setup-uv@v4&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; Build&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;        run&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; uv build&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; Publish to PyPI&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;        uses&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; pypa/gh-action-pypi-publish@release/v1&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;id-token: write&lt;/code&gt; is required here because GitHub needs permission to mint a short-lived OIDC token that proves this specific Actions job is who it claims to be. PyPI verifies that token directly with GitHub, no API key or password ever touches this workflow.&lt;/p&gt;
&lt;p&gt;You need to configure trusted publishing on the PyPI side once (under your project&#39;s Publishing settings), telling it which GitHub repo and workflow is allowed to publish. After that, pushing a tag like &lt;code&gt;v0.1.0&lt;/code&gt; runs tests across all target Python versions and publishes automatically if they pass. Works for both new and existing PyPI projects.&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>forcing positional and keyword arguments in a python function</title>
        <published>2026-04-03T00:00:00+00:00</published>
        <updated>2026-04-03T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/til/forcing-positional-and-keyword-arguments-in-a-python-function/"/>
        <id>https://emmanuelniyonshuti.github.io/til/forcing-positional-and-keyword-arguments-in-a-python-function/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/til/forcing-positional-and-keyword-arguments-in-a-python-function/">&lt;p&gt;consider this arbitrary function:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;def f(user_id: str, /, message, *, channel: str) -&amp;gt; bool:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    pass&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;/&lt;/code&gt; means everything to its left must be passed positionally. so &lt;code&gt;user_id&lt;/code&gt; cannot be passed as a keyword argument&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;f(&amp;quot;123&amp;quot;, &amp;quot;hello&amp;quot;, channel=&amp;quot;general&amp;quot;)     # ok&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;f(user_id=&amp;quot;123&amp;quot;, message=&amp;quot;hello&amp;quot;, channel=&amp;quot;general&amp;quot;)  # TypeError&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;message&lt;/code&gt; is a normal argument. you can pass it positionally or as a keyword.
&lt;code&gt;*&lt;/code&gt; means everything to its right must be passed as keyword arguments.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;f(&amp;quot;123&amp;quot;, &amp;quot;hello&amp;quot;, channel=&amp;quot;general&amp;quot;)     # ok&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;f(user_id=&amp;quot;123&amp;quot;, message=&amp;quot;hello&amp;quot;, channel=&amp;quot;general&amp;quot;)  # TypeError&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You violate these rules and python raises a TypeError &lt;code&gt;TypeError&lt;/code&gt;&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>SQLAlchemy Async ORM: The MissingGreenlet error after commit</title>
        <published>2026-03-31T00:00:00+00:00</published>
        <updated>2026-03-31T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/disable-expire-on-commit-with-sqlalchemy/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/disable-expire-on-commit-with-sqlalchemy/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/disable-expire-on-commit-with-sqlalchemy/">&lt;p&gt;&lt;em&gt;pre&lt;/em&gt;: This is not a concern for sqlalchemy core module user.&lt;/p&gt;
&lt;h3 id=&quot;tldr&quot;&gt;TLDR:&lt;/h3&gt;
&lt;p&gt;If you use the sqlalchemy async orm module and you try to access object attributes after committing a session, you will have to call &lt;code&gt;await session.refresh(your_object)&lt;/code&gt; for each object you want to access. if you forget, you will run into the greenlet error. setting &lt;code&gt;expire_on_commit=False&lt;/code&gt; is one way to avoid that.&lt;/p&gt;
&lt;h2 id=&quot;the-basics&quot;&gt;the basics&lt;/h2&gt;
&lt;p&gt;You need to first create an &lt;code&gt;engine&lt;/code&gt; that will manage the connection to the
relational database of your choice, the next step is to create a highlever interface to the database called a &lt;code&gt;session&lt;/code&gt;.
you bind this session to the engine you created, and you use this session to interact with your database.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; sqlalchemy&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; create_engine&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; sqlalchemy&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;orm&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; sessionmaker&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;engine&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; create_engine&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;sqlite:///mydatabase.db&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Session&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; sessionmaker&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;bind&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span&gt;engine&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;session&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; Session&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The next step to this is creating your database tables which is done by creating a &lt;code&gt;Base&lt;/code&gt; or &lt;code&gt;Model&lt;/code&gt; class that will hold the metadata for your tables(name of this class is up to you, those used are the most commonly used), and then you create your tables as classes that inherit from it. after you create your tables, you can use the &lt;code&gt;session&lt;/code&gt; to interact with your database.
Let&#39;s extend above code example to include the creation of a user table.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;...&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;class&lt;/span&gt;&lt;span style=&quot;color: #F9E2AF;font-style: italic;&quot;&gt; Model&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #F9E2AF;font-style: italic;&quot;&gt;DeclarativeBase&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    pass&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# creating a user table&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;class&lt;/span&gt;&lt;span style=&quot;color: #F9E2AF;font-style: italic;&quot;&gt; User&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #F9E2AF;font-style: italic;&quot;&gt;Model&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    __tablename__&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;users&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    uid&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; Column&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;Integer&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; primary_key&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;True&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; Column&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;String&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Usually, you will need to go a little bit further and configure the metadata for you tables in this &lt;code&gt;Model class&lt;/code&gt; by setting the &lt;code&gt;naming_conventions&lt;/code&gt; for your tables metadata so you can control the names of the &lt;code&gt;constraints&lt;/code&gt; and &lt;code&gt;indexes&lt;/code&gt; that will be generated for your tables, but lets not go into that in this article.&lt;/p&gt;
&lt;h2 id=&quot;async-orm&quot;&gt;async orm&lt;/h2&gt;
&lt;p&gt;That was for the most basic common sqlalchemy orm usage, but what if we want to use it async? what if we want to interact with our database in a non blocking way?
well, sqlalchemy orm module has an async version that you can use to interact with your database in a non blocking way. Though, we will need to modify our code a bit, we will need an &lt;code&gt;async database driver&lt;/code&gt;, Many database have them examples are  &lt;code&gt;asyncpg&lt;/code&gt; for postgres, &lt;code&gt;aiomysql&lt;/code&gt; for mysql and &lt;code&gt;aiosqlite&lt;/code&gt; for sqlite
SQLAlchemy&#39;s ORM is internally synchronous, and &lt;code&gt;greenlet&lt;/code&gt; is what lets it cooperate with asyncio. We will need to have it installed as well.
You can install sqlalchemy with its greenlet dependency by running:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;pip install&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; sqlalchemy&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;[&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;asyncio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;]&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s modify our code example to use &lt;code&gt;create_async_engine&lt;/code&gt;, and &lt;code&gt;async_sessionmaker&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; sqlalchemy&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;ext&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;asyncio&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; async_sessionmaker&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span&gt; create_async_engine&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; sqlalchemy&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;orm&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; DeclarativeBase&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span&gt; Mapped&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span&gt; mapped_column&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;engine&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; create_async_engine&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;sqlite+aiosqlite:///mydatabase.db&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Session&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; async_sessionmaker&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;bind&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span&gt;engine&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # expire_on_commit=True by default&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;session&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; Session&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;class&lt;/span&gt;&lt;span style=&quot;color: #F9E2AF;font-style: italic;&quot;&gt; Model&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #F9E2AF;font-style: italic;&quot;&gt;DeclarativeBase&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    pass&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;class&lt;/span&gt;&lt;span style=&quot;color: #F9E2AF;font-style: italic;&quot;&gt; User&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #F9E2AF;font-style: italic;&quot;&gt;Model&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    __tablename__&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;users&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    uid&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; Mapped&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;[&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;font-style: italic;&quot;&gt;int&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;]&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; mapped_column&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;primary_key&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;True&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    name&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; Mapped&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;[&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;font-style: italic;&quot;&gt;str&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;]&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; add_user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;name&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;font-style: italic;&quot;&gt; str&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;) -&amp;gt;&lt;/span&gt;&lt;span&gt; User&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    user&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; User&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span&gt;name&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    session&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;add&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    await&lt;/span&gt;&lt;span&gt; session&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;commit&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    return&lt;/span&gt;&lt;span&gt; user&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; create_tables&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    async with&lt;/span&gt;&lt;span&gt; engine&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;begin&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; as&lt;/span&gt;&lt;span&gt; conn&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;        await&lt;/span&gt;&lt;span&gt; conn&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;run_sync&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;Model&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;metadata&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;create_all&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; main&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    await&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; create_tables&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    user&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; await&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; add_user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;Foo&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;    print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;user&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;uid&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;if&lt;/span&gt;&lt;span&gt; __name__&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; ==&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;__main__&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    asyncio&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;main&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;())&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Running the code:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F38BA8;font-style: italic;&quot;&gt;...&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;raise&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; exc.MissingGreenlet&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F38BA8;font-style: italic;&quot;&gt;    ...&lt;/span&gt;&lt;span&gt;&amp;lt;2&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; line&lt;/span&gt;&lt;span&gt;s&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;...&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;    )&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;sqlalchemy.exc.MissingGreenlet:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; greenlet_spawn has not been called&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;;&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; can&amp;#39;t call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In our code above we tried to add a user and commit it to the database and return the user object, right after we commit, the session expires all the data in it by default. but, we try to access the &lt;code&gt;uid&lt;/code&gt; of the user object that we just added and since the session has expired, it will try to &lt;code&gt;refresh&lt;/code&gt; the data by querying the database again,
and this refresh attempt happens outside the greenlet context (the async context ended at commit), it raises the greenlet error.
To fix this, we can set the &lt;code&gt;expire_on_commit&lt;/code&gt; parameter to &lt;code&gt;False&lt;/code&gt; when we create our session, this will prevent the session from expiring the data after we commit it, and we will be able to access the &lt;code&gt;uid&lt;/code&gt; of the user object without any issues.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Session&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; async_sessionmaker&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;bind&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span&gt;engine&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; expire_on_commit&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;False&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;outuput of the code above will be:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;1&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We could have just called &lt;code&gt;await session.refresh(user)&lt;/code&gt; right after we committed the session to refresh the data in the session, but we will have to do that for every object we want to access its attributes, I think that is very easy to forget to do compared to setting &lt;code&gt;expire_on_commit&lt;/code&gt; to &lt;code&gt;False&lt;/code&gt;.&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>Accessing FastAPI Swagger docs offline</title>
        <published>2026-03-21T00:00:00+00:00</published>
        <updated>2026-03-21T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/accessing-fastapi-swagger-docs-offline/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/accessing-fastapi-swagger-docs-offline/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/accessing-fastapi-swagger-docs-offline/">&lt;p&gt;One of the things that make people including myself like FastAPI is its ability to generate
swagger API documentation automatically for you. However, you can only access these docs when
you run your application while connected to the internet. In this blog post I am going to explain
why that is and how you can overcome it.&lt;/p&gt;
&lt;p&gt;FastAPI is a Python web framework for building APIs and microservices. It is not only used for that —
you can do many things any web framework can do too. The reason we say it&#39;s for building APIs is
partly because it makes it very easy and fast to build them.&lt;/p&gt;
&lt;p&gt;When you spin up a FastAPI development server, you can access its swagger documentation at
&lt;code&gt;http://localhost:8000/docs&lt;/code&gt;. If you have used other web frameworks like Flask,
getting docs set up usually means reaching for an extra library or doing some manual configuration.
FastAPI comes with that out of the box.&lt;/p&gt;
&lt;p&gt;To do this, FastAPI utilizes Python type hints and Pydantic models to construct a JSON file that
conforms to &lt;a rel=&quot;external&quot; href=&quot;https://www.openapis.org/&quot;&gt;OpenAPI standards&lt;/a&gt;. This is the &lt;code&gt;openapi.json&lt;/code&gt; file you
see when you access &lt;code&gt;/docs&lt;/code&gt;. FastAPI knows how to parse this file, then serves an HTML page that
loads the Swagger UI — which is a JavaScript bundle, a CSS file, and a favicon. Those three files
are what your browser fetches from a CDN when you first open &lt;code&gt;/docs&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The answer is obvious! let&#39;s download those files from the CDN, put them in our project&#39;s static
folder, and make FastAPI skip the default docs and use ours instead.&lt;/p&gt;
&lt;p&gt;Let&#39;s start a FastAPI application. I will use &lt;code&gt;uv&lt;/code&gt;:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;mkdir&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; my_app&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F38BA8;font-style: italic;&quot;&gt;cd&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; my_app&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; add &amp;quot;fastapi[standard]&amp;quot;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;main.py&lt;/code&gt;:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; fastapi&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; FastAPI&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;app&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; FastAPI&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;@app&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;get&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;/hello&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; greeting&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    return&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt; {&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;msg&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;Hi👋&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;}&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now let&#39;s use &lt;code&gt;curl&lt;/code&gt; to download the files. I will first create a static folder in the root directory:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;mkdir&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; -p static/docs&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Then download the files:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;curl&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; -o static/docs/swagger-ui-bundle.js https://cdn.jsdelivr.net/npm/swagger-ui-dist@5/swagger-ui-bundle.js&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;curl&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; -o static/docs/swagger-ui.css https://cdn.jsdelivr.net/npm/swagger-ui-dist@5/swagger-ui.css&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;curl&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; -o static/docs/favicon.png https://fastapi.tiangolo.com/img/favicon.png&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We will disable the default docs so we can mount our own instead, then create a custom &lt;code&gt;/docs&lt;/code&gt;
route that overrides the function responsible for returning the Swagger UI HTML:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; fastapi&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; FastAPI&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span&gt; Request&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; fastapi&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;staticfiles&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; StaticFiles&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; fastapi&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;openapi&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;docs&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; get_swagger_ui_html&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;app&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; FastAPI&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;docs_url&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;None&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; redoc_url&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;None&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# mount the static files on the application&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;app&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;mount&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;/static/docs&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; StaticFiles&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;directory&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;static/docs&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;),&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; name&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;static_docs&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# create a custom /docs endpoint&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# overrides get_swagger_ui_html so it uses our local files instead of the CDN&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# include_in_schema=False because we don&amp;#39;t need this route to appear in our schema&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;@app&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;get&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;/docs&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; include_in_schema&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;False&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; custom_docs&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;request&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;&quot;&gt; Request&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    return&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; get_swagger_ui_html&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;        openapi_url&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span&gt;request&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;app&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;openapi_url&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;        title&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span&gt;request&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;app&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;title&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;        swagger_js_url&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;/static/docs/swagger-ui-bundle.js&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;        swagger_css_url&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;/static/docs/swagger-ui.css&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;        swagger_favicon_url&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;/static/docs/favicon.png&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;    )&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;@app&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;get&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;/hello&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;async def&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; greeting&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;():&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;    return&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt; {&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;msg&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;Hi👋&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;}&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That&#39;s all to it. One thing to keep in mind is that these CDN files can change depending on the
FastAPI version you are running. If you update FastAPI down the line, it is worth re-running those
&lt;code&gt;curl&lt;/code&gt; commands to pull the matching versions of the static files, otherwise your docs could silently
break or look off.&lt;/p&gt;
&lt;p&gt;Now I can access my swagger docs while offline. If you would rather use a package that handles this
for you, there is one on PyPI named &lt;code&gt;fastapi-offline&lt;/code&gt; it does the same thing under the hood,
but, with the static files bundled in at install time.&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>Fixing MissingGreenlet in Alembic with Async SQLAlchemy</title>
        <published>2026-02-16T00:00:00+00:00</published>
        <updated>2026-02-16T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/fixing-missinggreenlet-with-alembic-and-asyncio/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/fixing-missinggreenlet-with-alembic-and-asyncio/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/fixing-missinggreenlet-with-alembic-and-asyncio/">&lt;p&gt;If you&#39;re using sqlalchemy with async/await (asyncio) in your application - meaning you have an async engine created with &lt;code&gt;create_async_engine()&lt;/code&gt; and you&#39;re using an async database driver like asyncpg - and you tried to autogenerate your migration script with alembic and hit an error that says:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;sqlalchemy.exc.MissingGreenlet&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;and you are looking for how you can fix it, then you are at the right place. I am using postgresql database but this should be the same for other relational databases too. This error can be confusing, at least for me, because it doesn&#39;t tell us exactly what went wrong. Though the error is explained in sqlalchemy docs, I don&#39;t think it is explained for this particular situation with alembic migrations.&lt;/p&gt;
&lt;p&gt;In this article I am going to talk about how you can resolve this error in particular to the schema migration with alembic and get your schema autogenerations and upgrading to work.&lt;/p&gt;
&lt;h3 id=&quot;tldr&quot;&gt;TLDR&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;You could let alembic migrations stay synchronous by just using the synchronous database driver(psycopg2) for managing migrations which will demand you installing the database driver.*&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Instead of having to install psycopg2 just because of alembic migrations, you can make alembic use SQLAlchemy Async engine.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;We have an async_engine in the application.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;async_engine&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; = create_async_engine&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    url&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;f&amp;quot;postgresql+asyncpg://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}&amp;quot;,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    echo&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;True,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And we also have some tables in the application. I will define mine using python classes with sqlalchemy orm. I will first create declarative_base so the tables can inherit from it, and then create a table and name it &lt;code&gt;users&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;import&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; uuid&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;import&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; sqlalchemy as sa&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;import&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; sqlalchemy.orm as so&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;Base&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; = so.declarative_base&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;class&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; User&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;Base&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    __tablename__&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; = &amp;quot;users&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    uid:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; so.Mapped[str] = so.mapped_column&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;        sa.String,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; primary_key=True, default=lambda: str&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;uuid.uuid4&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;())&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;    )&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    email:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; so.Mapped[str] = so.mapped_column&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;sa.String,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; unique=True&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    password:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; so.Mapped[str] = so.mapped_column&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;sa.String&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;With this in place, I will initialize alembic&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;alembic&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; init alembic&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;After this command you get a complete environment for alembic. One thing we need to take care of is giving alembic our database connection url which it loads from the .ini file from a key &lt;code&gt;sqlalchemy.url&lt;/code&gt; by default and the Base metadata to know about tables in our application. We don&#39;t need alembic to read the database connection url from the .ini file because that can be a problem since this file is to be kept in the version control as well. So instead I need to find a way to bring the connection url inside the env.py and override the one it tries to read from the .ini &lt;code&gt;sqlalchemy.url&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;So, we will import the async_engine from the module we put it in into our alembic env.py file and we will override the key on the config object. Do this somewhere below &lt;code&gt;config = context.config&lt;/code&gt; in the env.py, so here we go:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;config.set_main_option(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    &amp;quot;sqlalchemy.url&amp;quot;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; async_engine.url.render_as_string&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;hide_password&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;False&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now I have overrode the database connection, but wait! This is not just the url! Well, if I only use the url string the password will be hidden and I will keep getting &lt;code&gt;sqlalchemy.exc.OperationalError&lt;/code&gt; with the error saying that the &lt;code&gt;password authentication failed&lt;/code&gt;. This is because sqlalchemy intentionally hides the password in the url by default.&lt;/p&gt;
&lt;p&gt;Now with that in place, we should be able to autogenerate our migration scripts, you might think or have thought!. But it won&#39;t work. This is the error that running &lt;code&gt;alembic revision --autogenerate -m&#39;users table&#39;&lt;/code&gt; gives me:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;  File&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;/home/emmanuel/lab/migration_tut/.venv/lib/python3.14/site-packages/sqlalchemy/util/_concurrency_py3k.py&amp;quot;, line 123, in await_only&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    raise&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; exc.MissingGreenlet&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F38BA8;font-style: italic;&quot;&gt;    ...&lt;/span&gt;&lt;span&gt;&amp;lt;2&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; line&lt;/span&gt;&lt;span&gt;s&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;...&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;    )&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;sqlalchemy.exc.MissingGreenlet:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; greenlet_spawn has not been called&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;;&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; can&amp;#39;t call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Strange! luckily, I have url I can head to for some help. If I head to the provided url I will see this:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;A call to the async DBAPI was initiated outside the greenlet spawn context usually setup by the SQLAlchemy AsyncIO proxy classes. Usually this error happens when an IO was attempted in an unexpected place, using a calling pattern that does not directly provide for use of the await keyword. When using the ORM this is nearly always due to the use of lazy loading, which is not directly supported under asyncio without additional steps and/or alternate loader patterns in order to use successfully.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;If you have a pretty good understanding of the inner workings of sqlalchemy with asyncio then you have a better understanding than me of why this error happens. For me I would presumably think that there is somewhere I am trying to perform something that is expected to be asynchronous but I am doing it synchronously. If that makes any sense, like there is something I am not awaiting for when I should be awaiting for it. But there isn&#39;t in the application, it&#39;s probably on alembic&#39;s side.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;There are two ways we can solve this error&lt;/strong&gt;. One is we can let alembic handle migration synchronously as it normally does, we can use synchronous database driver  like psycopg instead. In this way, since the postgresql dialect we used defaults to using psycopg2 driver, we can update our &lt;code&gt;env.py&lt;/code&gt; file to do this:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;config.set_main_option(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    &amp;quot;sqlalchemy.url&amp;quot;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; async_engine.url.render_as_string&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;hide_password&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;False&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;.replace&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;        &amp;quot;postgresql+asyncpg&amp;quot;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;postgresql&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You can explicitly replace &lt;code&gt;&quot;postgresql+asyncpg&quot;&lt;/code&gt; with    &lt;code&gt;&quot;postgresql+psycopg2&quot;&lt;/code&gt; but I did not have to because again psycopg2 is the default driver for postgresql dialect in python. With that in place, we then have to install &lt;code&gt;psycopg2&lt;/code&gt; in our virtual environment. I recommend you just install &lt;code&gt;psycopg2-binary&lt;/code&gt; binary package which comes pre-compiled and you will not need to install anything extra on the system.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;pip install psycopg2-binary&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;# uv&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;uv add psycopg2-binary&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If for some reason you want to install the source version psycopg2 instead, you&#39;ll need to install some build dependencies first. On Ubuntu which I use, you&#39;d run:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;sudo&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; apt update&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;sudo&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; apt install libpq-dev python3-dev build-essential&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Then:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;pip&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; install psycopg2&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# or with uv&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; add psycopg2&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At this point, we are good to go. I can autogenerate the migration scripts and upgrade the database now.
But, that is one way to do it. Another way is, we can make alembic use SQLAlchemy Async engine. As of now, Alembic doesn&#39;t provide an async API directly, but it does provide this option in its docs &lt;a rel=&quot;external&quot; href=&quot;https://alembic.sqlalchemy.org/en/latest/cookbook.html#using-asyncio-with-alembic&quot;&gt;using-asyncio-with-alembic&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;To do this, we need to update our env.py file to use asyncio. I will place the code here, but it&#39;s all from the alembic docs. We will need to replace the &lt;code&gt;run_migrations_online()&lt;/code&gt; function in env.py and everything else stays the same.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;from&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; sqlalchemy.ext.asyncio import async_engine_from_config&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;import&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; asyncio&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;def&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; do_run_migrations&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;connection&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    context.configure(connection&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;=connection, target_metadata=target_metadata&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    with&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; context.begin_transaction&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;        context.run_migrations&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;async&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; def run_async_migrations&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    &amp;quot;&amp;quot;&amp;quot;In this scenario we need to create an Engine&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    and associate a connection with the context.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    &amp;quot;&amp;quot;&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    connectable&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; = async_engine_from_config&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;        config.get_section(config.config_ini_section,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; {}&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        prefix&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;sqlalchemy.&amp;quot;,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        poolclass&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;pool.NullPool,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    )&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    async&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; with connectable.connect&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; as connection:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;        await&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; connection.run_sync&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;do_run_migrations&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    await&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; connectable.dispose&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;def&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; run_migrations_online&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    &amp;quot;&amp;quot;&amp;quot;Run migrations in &amp;#39;online&amp;#39; mode.&amp;quot;&amp;quot;&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    asyncio.run(run_async_migrations(&lt;/span&gt;&lt;span&gt;))&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;if&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; context.is_offline_mode&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    run_migrations_offline&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;else:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    run_migrations_online&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;</content>
        
    </entry>
    <entry xml:lang="en">
        <title>Externally-managed Python Environments and uv</title>
        <published>2026-01-16T00:00:00+00:00</published>
        <updated>2026-01-16T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/externally-managed-python-environments/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/externally-managed-python-environments/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/externally-managed-python-environments/">&lt;p&gt;Python environment isolation is one of the important parts of working with Python. In this article I&#39;m going to be talking about externally-managed Python environments, uv-managed Python environments, and some other things that revolve around Python package locations and virtual environments.
I got prompted to write this article mostly because of an error you hit when you try to install a pip package system-wide, but I couldn&#39;t find a way to just write only about it. So I thought that it would be much better to instead talk about Python environments in a broader sense so that you can solve  problems like that when you encounter one.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;A quick note&lt;/strong&gt;: I primarily use Ubuntu as my operating system, so this article is written from that perspective. It should apply to other Linux distributions, though I might be off on a few points since this is a broader topic. And more importantly, this reflects my current understanding of it, hence there may be things I might skip or overlook.&lt;/p&gt;
&lt;h3 id=&quot;externally-managed-environment&quot;&gt;externally-managed-environment&lt;/h3&gt;
&lt;p&gt;You hit this when you try to install a package system-wide with pip on your Linux machine.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;emmanuel@LAPTOP-A0O3BKPI:~$&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; pip install ruff&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;error:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; externally-managed-environment&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;×&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; This environment is externally managed&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;╰─&lt;/span&gt;&lt;span&gt;&amp;gt;&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; To install Python packages system-wide, try apt install&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    python3-xyz,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; where xyz is the package you are trying to&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;    install.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This error tells us that the environment we&#39;re trying to install our package in is externally managed, meaning it&#39;s managed by the operating system. So we should use the package manager for it, like apt for example, to install that package so it won&#39;t corrupt the system&#39;s packages. This is basically &lt;a rel=&quot;external&quot; href=&quot;https://peps.python.org/pep-0668/&quot;&gt;PEP 668&lt;/a&gt;, which was introduced a couple years ago and implemented in Python 3.11.&lt;/p&gt;
&lt;p&gt;Using the OS package manager is better because it&#39;s more aware than pip, so it won&#39;t corrupt packages that your OS might be depending on. The operating system - Linux distro in this case - has its own Python, or system Python. It relies on this Python installation for some of the tools it uses. Some might even be the very tool you might be attempting to install sometimes, hence, it warns us to not corrupt it.&lt;/p&gt;
&lt;p&gt;So, how do you solve this? Using virtual environments for your packages is the more intuitive way I guess. But, I think in most cases you don&#39;t run into this error because you forgot virtual environments exist. Rather, it&#39;s when you want to install a CLI tool system-wide.&lt;/p&gt;
&lt;p&gt;For command line tools - say you want to install black or ruff, tools you don&#39;t import in your code - you use &lt;code&gt;pipx install &amp;lt;package&amp;gt;&lt;/code&gt;. Or if you use uv, &lt;code&gt;uv tool install &amp;lt;package&amp;gt;&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;You might see the error message suggesting the use of apt to install Python packages like &lt;code&gt;python3-requests&lt;/code&gt;. While you can install packages this way, they are intended for system-level dependencies required by Ubuntu. For your own development, you should not install Python packages through apt; instead, it is advised to use virtual environments for projects and tools like pipx or uv for CLI tools.&lt;/p&gt;
&lt;p&gt;In my case when I hit this error, I should have used &lt;code&gt;uv tool install &amp;lt;package&amp;gt;&lt;/code&gt; instead of trying to pip install it system-wide.&lt;/p&gt;
&lt;h3 id=&quot;dist-packages-vs-site-packages&quot;&gt;dist-packages vs site-packages&lt;/h3&gt;
&lt;p&gt;Debian-based Linux distributions like Ubuntu modify Python package locations. Instead of using the standard site-packages, they use dist-packages.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;emmanuel@LAPTOP-A0O3BKPI:~$&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; ls /usr/lib/python3&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;dist-packages&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You&#39;ve probably seen site-packages in error tracebacks - file paths often point into it. That&#39;s where Python looks for installed packages. Basically, when you start a Python interpreter, it runs a module called &lt;code&gt;site.py&lt;/code&gt;. This module sets up Python&#39;s module search path &lt;code&gt;sys.path&lt;/code&gt;, which is an array of string paths - those are the paths where Python searches for modules you import in your code. Debian modifies this to add dist-packages directories to the list so that system Python will search for modules in that dist-packages directory as well.&lt;/p&gt;
&lt;p&gt;Debian based linux distributions like Ubuntu uses Python for its core tools, and those tools rely on specific Python packages. If we install or upgrade packages with pip that the OS depends on, we might break things on the system. hence, they keep them separate:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;dist-packages&lt;/code&gt; for OS-managed packages (those installed with apt)&lt;/li&gt;
&lt;li&gt;&lt;code&gt;site-packages&lt;/code&gt; for user-installed packages (those installed with pip)&lt;/li&gt;
&lt;li&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;There&#39;s a blog that goes into this really well if you want to learn more about it: &lt;a rel=&quot;external&quot; href=&quot;https://ffy00.github.io/blog/02-python-debian-and-the-install-locations/&quot;&gt;Filipe Laíns - Python, Debian, and the install locations&lt;/a&gt;. I&#39;m not going to explain all the details because I don&#39;t fully understand them myself, but I think this is the general idea.&lt;/p&gt;
&lt;p&gt;I think Python seems to have somehow adopted what Debian was doing rather than the other way around. Debian was modifying Python to protect their system, and then Python itself introduced PEP 668 to formalize this protection. Modern Python tools like uv now embrace this isolation approach even more.&lt;/p&gt;
&lt;h3 id=&quot;system-packages-and-virtual-environments&quot;&gt;System packages and virtual environments&lt;/h3&gt;
&lt;p&gt;In some cases you might have packages installed system-wide and don&#39;t want to reinstall them in every project virtual environment. There is a way you can bridge your virtual environment to access system packages. You can set &lt;code&gt;include-system-site-packages = true&lt;/code&gt; in your venv&#39;s &lt;code&gt;pyvenv.cfg&lt;/code&gt;.
But this only works if you created the venv with your operating system&#39;s Python. You can check which Python created the venv, you find that in &lt;code&gt;pyvenv.cfg&lt;/code&gt; under the &lt;code&gt;home&lt;/code&gt; key.&lt;/p&gt;
&lt;p&gt;If you&#39;re using uv and created the venv with a uv-managed Python, &lt;code&gt;include-system-site-packages&lt;/code&gt; won&#39;t pull in Ubuntu&#39;s apt-installed packages. With uv you won&#39;t be able to access those system-wide packages because uv-managed Python environments don&#39;t mix system-wide packages with packages installed inside the uv environment.&lt;/p&gt;
&lt;p&gt;While you can find ways around this, it&#39;s encouraged to instead use virtual environments for development packages and keep things isolated.&lt;/p&gt;
&lt;h3 id=&quot;uv-and-modern-python-tooling&quot;&gt;uv and modern Python tooling&lt;/h3&gt;
&lt;p&gt;uv is a python package and project manager written in Rust that has gained huge traction in the Python ecosystem.
If you have not used uv before, you most likely have  used or use tools like pip for installing packages. Well, uv does everything you do with pip plus the enhanced performance and combinations of all other things like virtual environment management, project dependency locking,  managing different versions of python, etc. which with pip you normally do separately . uv consolidate all of these  into one tool with enhanced speed and simplicity of using it.&lt;/p&gt;
&lt;p&gt;I&#39;ll touch on the basics of getting started with uv since it relates to what we&#39;re  talking about.
Installing uv is straightforward. On Linux or macOS:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;curl&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; -LsSf https://astral.sh/uv/install.sh&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; |&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; sh&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Once you have that, here are a few things you can do:&lt;/p&gt;
&lt;p&gt;Installing a CLI tool globally(ruff as my example!):&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; tool install ruff&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Running a Python script with a specific version without installing it:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; run --python&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 3.12&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; script.py&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Creating a new project with dependencies:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; init my-project&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F38BA8;font-style: italic;&quot;&gt;cd&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; my-project&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; add requests&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;I&#39;m not going to explain most of the things you can do with uv because the &lt;a rel=&quot;external&quot; href=&quot;https://docs.astral.sh/uv/&quot;&gt;documentation&lt;/a&gt; does that really well, and honestly these LLM tools can help quickly  if you forget certain commands. you can also copy paste some certain parts of the documentation for the llm to demystify it for you.&lt;/p&gt;
&lt;p&gt;Anyways, uv seems to embrace the whole isolation thing that PEP 668 is about. When you use uv to run something or install a package for a project, it automatically creates a virtual environment if one doesn&#39;t exist. you can see in the commands above there is nowhere I run &lt;code&gt;python3 -m venv &amp;lt;my_virtualenvironmant_name&amp;gt;&lt;/code&gt; because uv takes care of that, but you can also create it if you want to by running &lt;code&gt;uv venv&lt;/code&gt; .&lt;/p&gt;
&lt;h3 id=&quot;uv-managed-python-installations&quot;&gt;uv-managed Python installations&lt;/h3&gt;
&lt;p&gt;uv can also manage different Python versions for you. Most people including myself used to use &lt;code&gt;pyenv&lt;/code&gt; for this. pyenv is quite a mature tool now and works well, but it requires compilation and can be slow to install new Python versions compared to uv.&lt;/p&gt;
&lt;p&gt;with uv you can install a Python version with:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; python install&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 3.14&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And list what you have:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; python list&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;I have Python 3.12 from Ubuntu, plus Python 3.8 all the way up to 3.14 managed by uv. I barely use some of them really.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;emmanuel@LAPTOP-A0O3BKPI:~$&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; which python3&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt; # system python executable location&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;/usr/bin/python3&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;emmanuel@LAPTOP-A0O3BKPI:~$&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; python3 --version&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;Python&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 3.12.3&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;emmanuel@LAPTOP-A0O3BKPI:~$&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; ls /home/emmanuel/.local/share/uv/python/&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;cpython-3.10.18-linux-x86_64-gnu&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;  cpython-3.13.6-linux-x86_64-gnu     cpython-3.9.23-linux-x86_64-gnu&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;cpython-3.11.13-linux-x86_64-gnu&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;  cpython-3.14.2-linux-x86_64-gnu&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;cpython-3.12.11-linux-x86_64-gnu&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;  cpython-3.8.20-linux-x86_64-gnu&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;emmanuel@LAPTOP-A0O3BKPI:~$&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;They&#39;re completely separate installations. You can see in the path too. Each one has its own &lt;code&gt;site-packages&lt;/code&gt; directory:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;emmanuel@LAPTOP-A0O3BKPI:~$&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; ls /home/emmanuel/.local/share/uv/python/cpython-3.14.2-linux-x86_64-gnu/lib/python3.14/site-packages/&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;README.txt&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;  pip  pip-25.3.dist-info&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;emmanuel@LAPTOP-A0O3BKPI:~$&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;When I create a venv with uv&#39;s Python and check &lt;code&gt;sys.path&lt;/code&gt;, it only picks up uv&#39;s own site-packages. That seems intentional. From what I understand, uv focuses on isolated environments and explicit dependencies, without mixing in system packages.&lt;/p&gt;
&lt;h3 id=&quot;python-3-14-does-something-different&quot;&gt;Python 3.14 does something different&lt;/h3&gt;
&lt;p&gt;One point I can&#39;t leave out is that Python 3.14 does something different. In Python 3.13 and earlier, the &lt;code&gt;site&lt;/code&gt; module was responsible for detecting virtual environments and updating &lt;code&gt;sys.prefix&lt;/code&gt; to point to the venv.&lt;/p&gt;
&lt;p&gt;Python 3.14 moved this earlier. Now it happens during path initialization, before &lt;code&gt;site.py&lt;/code&gt; even runs. So &lt;code&gt;sys.prefix&lt;/code&gt; points to your venv even if you start Python with &lt;code&gt;-S&lt;/code&gt; (which disables site).&lt;/p&gt;
&lt;p&gt;I think this is part of making virtual environment detection more fundamental to Python itself, rather than something that happens in a module that can be disabled.&lt;/p&gt;
&lt;h3 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h3&gt;
&lt;p&gt;There&#39;s a lot more to Python environments, Python packaging and module search than what I covered here. I touched on several things that relate to each other - externally-managed environments, dist-packages, uv - rather than focusing on just one. Maybe I&#39;ll write about using uv specifically in another article. Ultimately, I hope you learned something from this.&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>Docker Containers: FastAPI + PostgreSQL</title>
        <published>2026-01-02T00:00:00+00:00</published>
        <updated>2026-01-02T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/docker-containers-as-my-development-environment-fastapi-postgresql-1/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/docker-containers-as-my-development-environment-fastapi-postgresql-1/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/docker-containers-as-my-development-environment-fastapi-postgresql-1/">&lt;p&gt;The way I started using Docker was probably not very common. Let me give you some context about how I got into it.
In 2024, I was learning software engineering at ALX SE, where Docker was an optional tool for working on projects.
In most cases, I didn&#39;t actually use it. However, we were required to run some of our projects on Python 3.8, Node 18 (I think), and MySQL (I don&#39;t remember the exact version), and sometimes these versions didn&#39;t match what I already had on my machine.
When we started, I didn&#39;t bother much with tools to manage different versions of these software packages.
I did discover pyenv along the way and used it a couple of times if you don&#39;t know pyenv, it&#39;s a tool for managing different Python versions (this was before uv). I also used nvm (a Node version manager) occasionally, but I think at some point I might have gotten bored or frustrated with the process and nvm seemed much more straightforward to me than pyenv!
Since I knew and had read a little about Docker containers, I started using docker containers instead. so, instead of trying to switch between different versions, I would pull an image from Docker Hub for the specific version of Python or MySQL I needed, volume mount my project directory (I&#39;ll talk about volumes in this article) to the container, and then run my code in the container environment. I know we can chalk that up to a skill issue at some point, I think, but it kind of solved my problem for some time as my code would pass their automated checker. The thing is, it was one of those things where you feel like you get the hang of it even though it still feels like a magic.&lt;/p&gt;
&lt;p&gt;In this article, I&#39;m going to talk about how to actually use Docker containers as your development environment and potentially deploy with them.
To work with Docker containers, you have to get comfortable with two main concepts:&lt;strong&gt;Images&lt;/strong&gt; and &lt;strong&gt;Containers&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The Image&lt;/strong&gt;: This is a complete package or blueprint of your application. When using Docker, you deal with images by either pulling them from Docker Hub, which is a repository containing pre-built and ready-to-use images, or by building one yourself by writing a Dockerfile.
&lt;strong&gt;The Container&lt;/strong&gt;: This is the image when it&#39;s actually running. You run a container from a container image.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;A basic workflow is&lt;/em&gt;: you write a Dockerfile, you build an image from that file, and then you run a container from that image.
Everything I talk about in this article revolves around those two concepts.&lt;/p&gt;
&lt;p&gt;In this article I will use a simple user authenticaion REST API built with fastapi using postgresql as database.
The complete code I used is available here: &lt;a rel=&quot;external&quot; href=&quot;https://github.com/EmmanuelNiyonshuti/fastapi-uv-docker-template&quot;&gt;fastapi-uv-docker-template&lt;/a&gt;&lt;/p&gt;
&lt;h3 id=&quot;my-project-structure&quot;&gt;My Project Structure&lt;/h3&gt;
&lt;p&gt;on my computer this is how my project looks like :&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;emmanuel@LAP-01:~/web-backend$&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; tree --prune -I &amp;quot;__pycache__|*.pyc&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F38BA8;font-style: italic;&quot;&gt;.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;├──&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; Dockerfile&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;├──&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; README.md&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;├──&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; alembic&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   ├── README&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   ├── env.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   ├── script.py.mako&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   └── versions&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;       └── 7c0c48b52736_add_users_table.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;├──&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; alembic.ini&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;├──&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; app&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   ├── __init__.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   ├── api&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   │   ├── __init__.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   │   ├── deps.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   │   ├── main.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   │   └── routes&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   │       ├── __init__.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   │       └── user.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   ├── core&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   │   ├── __init__.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   │   ├── config.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   │   └── db.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   ├── main.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;│&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;   └── models.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;├──&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; docker-compose.yml&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;├──&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; pyproject.toml&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;└──&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; uv.lock&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;7&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; directories,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 21&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; files&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;emmanuel@LAP-01:~/web-backend$&lt;/span&gt;&lt;span&gt; &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;I used the &lt;a rel=&quot;external&quot; href=&quot;https://linux.die.net/man/1/tree&quot;&gt;tree&lt;/a&gt; command with some flag to hide Python cache files.
It&#39;s a basic FastAPI project with uv as the package manager, PostgreSQL as the database, and SQLAlchemy as the orm.
This is a setup that made the most sense to me as I can develop and potentially deploy everything through Docker containers.&lt;/p&gt;
&lt;h3 id=&quot;the-dockerfile-building-my-app-image&quot;&gt;The Dockerfile - Building My App Image&lt;/h3&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;docker&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;FROM&lt;/span&gt;&lt;span&gt; python:3.12-slim&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;WORKDIR&lt;/span&gt;&lt;span&gt; /web-backend&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Install uv&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;COPY&lt;/span&gt;&lt;span&gt; --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Set environment variables&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;ENV&lt;/span&gt;&lt;span&gt; PYTHONDONTWRITEBYTECODE=1 \&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    PYTHONUNBUFFERED=1&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Copy dependency files first&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;COPY&lt;/span&gt;&lt;span&gt; pyproject.toml uv.lock ./&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Install dependencies&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;RUN&lt;/span&gt;&lt;span&gt; uv sync --frozen --no-cache --no-dev&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;ENV&lt;/span&gt;&lt;span&gt; PYTHONPATH=/web-backend&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;COPY&lt;/span&gt;&lt;span&gt; ./app ./app&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;COPY&lt;/span&gt;&lt;span&gt; ./alembic ./alembic&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;COPY&lt;/span&gt;&lt;span&gt; ./alembic.ini ./&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;CMD&lt;/span&gt;&lt;span&gt; [&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;uv&amp;quot;&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;run&amp;quot;&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;fastapi&amp;quot;&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;run&amp;quot;&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;app/main.py&amp;quot;&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;--port&amp;quot;&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;8000&amp;quot;&lt;/span&gt;&lt;span&gt;]&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;strong&gt;FROM python:3.12-slim&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The first thing you do is pulling the base image of your application. in this case we have a python application so our base image will be python.
Docker has pre-built and ready to use images that you can use.
I start with an official Python image that already has Python 3.12 installed. The &lt;code&gt;slim&lt;/code&gt; variant is smaller debian based image. but you can choose other variants as well.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;WORKDIR /web-backend&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;setting the working directory to &lt;code&gt;/web-backend&lt;/code&gt; and creates it if it doesn&#39;t exist. Every command after this runs from &lt;code&gt;/web-backend&lt;/code&gt;. I chose &lt;code&gt;/web-backend&lt;/code&gt; but you could use &lt;code&gt;/app&lt;/code&gt; or  whatever makes sense to you.The important thing is that this becomes the root of your project in the container, similar to how &lt;code&gt;~/web-backend&lt;/code&gt; is the root on my local machine. I like to have my container almost mirroring the local project structure.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;docker&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;COPY&lt;/span&gt;&lt;span&gt; --from=ghcr.io/astral-sh/uv:latest  /uv /uvx /bin/**&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This line copies the uv program from another Docker image into mine. Instead of installing uv by downloading and running an install script, we just copying an already-compiled uv binary directly. The format is &lt;code&gt;COPY --from=source destination&lt;/code&gt;. So I&#39;m copying &lt;code&gt;/uv&lt;/code&gt; and &lt;code&gt;/uvx&lt;/code&gt; from the &lt;code&gt;ghcr.io/astral-sh/uv:latest&lt;/code&gt; image and putting them in &lt;code&gt;/bin/&lt;/code&gt; in my container. The uv team provides these pre-built binaries in docker images for some benefits including overall smaller image size and improved speed than building uv from the source.&lt;/p&gt;
&lt;p&gt;You can also pin to a specific version instead of using &lt;code&gt;latest&lt;/code&gt;:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;docker&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;COPY&lt;/span&gt;&lt;span&gt; --from=ghcr.io/astral-sh/uv:0.9.21 /uv /uvx /bin/&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;docker&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;ENV&lt;/span&gt;&lt;span&gt; PYTHONDONTWRITEBYTECODE=1 and PYTHONUNBUFFERED=1**&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These are Python-specific optimizations. &lt;code&gt;PYTHONDONTWRITEBYTECODE&lt;/code&gt; prevents Python from creating &lt;code&gt;.pyc&lt;/code&gt; files  &lt;code&gt;PYTHONUNBUFFERED&lt;/code&gt; makes Python print output immediately instead of buffering it, which means I see my logs in real-time when running &lt;code&gt;docker-compose logs&lt;/code&gt;. there are some others you can add but those are just what I have for now.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;docker&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;**COPY pyproject.toml uv.lock ./**&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;copying the project dependency files, when you are working with uv as your package manager you don&#39;t need requirements.txt as you would do with pip. instead you have those two files which contains all of the project dependencies and their locked version as well. but we copied them first before the application code, a thing to know about is that when Docker build the image it will cache instructions in layers and these dependency files don&#39;t really change that often so, the next time we rebuild the container image instead of docker trying to install the dependencies again if we did not add any extra dependencies it will reuse the cache.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;docker&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;RUN&lt;/span&gt;&lt;span&gt; uv sync --frozen --no-cache**&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;this is a &lt;code&gt;uv sync&lt;/code&gt; command that is being executed.  this command ensures virtual environment exactly matches the dependencies listed in uv.lock file. The &lt;code&gt;--frozen&lt;/code&gt; flag means uv won&#39;t update the lock file&lt;code&gt;uv.lock&lt;/code&gt;. The &lt;code&gt;--no-cache&lt;/code&gt; flag will tell uv not to cache downloaded packages (Docker&#39;s layer caching handles this better).
uv creates a virtual environment at &lt;code&gt;/web-backend/.venv&lt;/code&gt; automatically in the container&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;docker&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;ENV&lt;/span&gt;&lt;span&gt; PYTHONPATH=/web-backend**&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;this is to enable imports like &lt;code&gt;from app.models import User&lt;/code&gt;.  with this Python will know that&lt;code&gt;/web-backend&lt;/code&gt; is the root of the project.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;docker&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;COPY&lt;/span&gt;&lt;span&gt; ./app ./app and COPY ./alembic ./alembic**&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now I copy my actual application code and alembic for database migrations management.. If I modify a Python file, only this layer and everything after it gets rebuilt. All the previous layers (including the slow dependency installation) use the cache.&lt;/p&gt;
&lt;p&gt;Notice I&#39;m copying &lt;code&gt;./app&lt;/code&gt; on my computer to &lt;code&gt;./app&lt;/code&gt; in the container (which becomes &lt;code&gt;/web-backend/app&lt;/code&gt; because my WORKDIR is &lt;code&gt;/web-backend&lt;/code&gt;).&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;docker&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;CMD&lt;/span&gt;&lt;span&gt; [&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;uv&amp;quot;&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;run&amp;quot;&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;fastapi&amp;quot;&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;run&amp;quot;&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;app/main.py&amp;quot;&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;--port&amp;quot;&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;8000&amp;quot;&lt;/span&gt;&lt;span&gt;]**&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is the default command that runs when the container starts. I&#39;m using the exec form (the array syntax) instead of the shell form because it handles signals properly.
FastAPI &lt;a rel=&quot;external&quot; href=&quot;https://fastapi.tiangolo.com/deployment/docker/#use-cmd-exec-form&quot;&gt;recommends&lt;/a&gt; using this exec form too.
in development I&#39;ll override this in docker-compose to use &lt;code&gt;fastapi dev&lt;/code&gt; for auto-reload.  command instruction in docker compose service takes precedence over this.&lt;/p&gt;
&lt;h3 id=&quot;the-dockerignore-file&quot;&gt;The .dockerignore File&lt;/h3&gt;
&lt;p&gt;Before building the image, Docker needs to know what files to ignore. This is like &lt;code&gt;.gitignore&lt;/code&gt; but for Docker:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;__pycache__&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;*.pyc&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;.env&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;.venv&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;.git&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;preventing  Docker from copying unnecessary files into the build context, which speeds up builds too. we definitely don&#39;t want to copy our local &lt;code&gt;.venv&lt;/code&gt; or &lt;code&gt;.env&lt;/code&gt; file into the image!&lt;/p&gt;
&lt;p&gt;with this in place now I can build container image&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;docker build -t web-image .&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;the above command will build a container image with a name web-image from Dockerfile in the current directory. after that you can check if your images with &lt;code&gt;docker images&lt;/code&gt; . with that container image in place I will have to run another command to actually run a container from it. &lt;code&gt;docker run -d web-image&lt;/code&gt; will actually run a container from that image -d is optional and it make it run in the background.&lt;/p&gt;
&lt;p&gt;with this in place I can keep developing while my application is running in the container. but, as we have the database the application talks to we need more than one container for this application. we do this with docker compose. docker compose allow us to run multi container applications like this one.&lt;/p&gt;
&lt;h3 id=&quot;docker-compose&quot;&gt;Docker Compose&lt;/h3&gt;
&lt;p&gt;we have two services: FastAPI app and PostgreSQL. to use docker compose we will write single YAML (&lt;code&gt;compose.yaml or docker-compose.yml&lt;/code&gt;) file to configure this entire application stack.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;yaml&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;services&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;  web&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    build&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; .&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    ports&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;8000:8000&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    env_file&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; .env&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    environment&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; POSTGRES_SERVER=db&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt; # will intercept the value from .env&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; POSTGRES_PORT=5432&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    depends_on&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;      db&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;        condition&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; service_healthy&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;        restart&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; true&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    volumes&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; ./app:/web-backend/app&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; ./alembic:/web-backend/alembic&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    command&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; uv run fastapi dev app/main.py --host 0.0.0.0 --port 8000&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;  db&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    image&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; postgres:17&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    env_file&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; .env&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    ports&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;5433:5432&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    volumes&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; postgres_data:/var/lib/postgresql/data&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    healthcheck&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;      test&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt; [&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;CMD-SHELL&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; &amp;quot;pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;]&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;      interval&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; 10s&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;      timeout&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; 5s&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;      retries&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 5&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;volumes&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;  postgres_data&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;in this file we have two services (web and db)
&lt;strong&gt;web service (my FastAPI app)&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;code&gt;build: .&lt;/code&gt; tells Docker to build an image from the Dockerfile in the current directory. Unlike the db service which uses a pre-built image, my web service needs to be built from my custom Dockerfile that we already have.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;ports: - &quot;8000:8000&quot;&lt;/code&gt; maps port 8000 on my computer to port 8000 in the container. The format is &lt;code&gt;HOST:CONTAINER&lt;/code&gt;. So when I visit &lt;code&gt;localhost:8000&lt;/code&gt; on my browser, it will forwards to port 8000 inside the container.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;env_file: - .env&lt;/code&gt; loads environment variables from my &lt;code&gt;.env&lt;/code&gt; file. This is cleaner than hardcoding them in the compose file. My &lt;code&gt;.env&lt;/code&gt; looks like this:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;POSTGRES_USER&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; emmanuel&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;POSTGRES_PASSWORD&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; super_secret&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;POSTGRES_DB&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; dev_db&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;POSTGRES_SERVER&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; localhost&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;POSTGRES_PORT&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt; 5433&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;SECRET_KEY&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;dev-secret-key&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;environment:&lt;/code&gt;
If you look at the environment variables in the &lt;code&gt;.env file&lt;/code&gt; we have &lt;code&gt;POSTGRES_SERVER= localhost&lt;/code&gt; and &lt;code&gt;POSTGRES_PORT= 5433&lt;/code&gt; but inside the web service we changed them to &lt;code&gt;POSTGRES_SERVER= db&lt;/code&gt; and&lt;code&gt; POSTGRES_PORT= 5432&lt;/code&gt;. I did this to override those environment variables in the container so that web container can communicate with db container. there are more ways you can achieve what this though but this why I do override them so essentially, each of these two services &lt;code&gt;web&lt;/code&gt; and &lt;code&gt;db&lt;/code&gt; is running in its own container. If I let the FastAPI application in the web container use localhost and port 5433, it won&#39;t find the database.
That localhost:5433 setup works on the local machine (like when I run migrations from my terminal) because on my computer, localhost means my actual computer, and port 5433 is the &quot;gate&quot; that leads into the container.
But, inside the container, localhost means the container itself. So if the web app looks at localhost, it is looking at its own container, not the database container. In a Docker Compose network, containers don&#39;t use localhost to talk to each other, they use the service name. In our particular case, the service name is db and it is running on port 5432 inside the network.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;depends_on&lt;/code&gt; with &lt;code&gt;condition: service_healthy&lt;/code&gt; means the web container won&#39;t start until the &lt;code&gt;db&lt;/code&gt; container passes its health check. Without this,  FastAPI app might try to connect to the database before PostgreSQL is ready and it would crash.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;volumes&lt;/code&gt; - this is very important during development and this is by far one of my favorite feature of Docker. Each line will create a live link between my computer and the container. meaning when I edit a file in my editor, the change appears instantly in the container. FastAPI&#39;s dev mode will see the change and auto-reloads.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;yaml&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    volumes&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; ./app:/web-backend/app&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; ./alembic:/web-backend/alembic&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The format is &lt;code&gt;HOST_PATH:CONTAINER_PATH&lt;/code&gt;. So my &lt;code&gt;./app&lt;/code&gt; directory on my computer is mounted to &lt;code&gt;/web-backend/app&lt;/code&gt; in the container.
I also mounted alembic as well, this is because I might do somethings in the migration scripts.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;command&lt;/code&gt; overrides the CMD in my Dockerfile. In development I want &lt;code&gt;fastapi dev&lt;/code&gt; for auto-reload and benefits with development server, the &lt;code&gt;fastapi run&lt;/code&gt; in Dockerfile is optimized for production environment.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;db service (PostgreSQL)&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;code&gt;image: postgres:17&lt;/code&gt; uses the official PostgreSQL image from Docker Hub. I don&#39;t need a custom Dockerfile for this because PostgreSQL provides everything I need.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;env_file: - .env&lt;/code&gt;
It will load the same variables I listed above in the web service. Normally, when you are not developing with docker containers, you have to have some  PostgreSQL database running on your computer or may be somewhere else, log into it, and write SQL statements to create the user for your application or use the existing one, create the database for your application, and grant permissions for that user to this database or you write a sql script file and execute it against the database and it does all that. either way you do it, you take those credentials and put them in your app.&lt;/p&gt;
&lt;p&gt;But here with Docker, we are just giving it the environment variables. These pre-built database image knows how to do all that work for us. I don&#39;t have to manually go into the database to set everything up; the Postgres image has a built-in script that runs the very first time the container starts.  It looks for variables &lt;code&gt;POSTGRES_USER&lt;/code&gt;, &lt;code&gt;POSTGRES_PASSWORD&lt;/code&gt;, and &lt;code&gt;POSTGRES_DB&lt;/code&gt;, and automatically creates that user and that database for me. This automatic setup only happens the very first time the container starts. After that, the database is stored in the volume, so even if I restart the container, my data and users are already there and waiting. and if I stop, delete or restart the db container those data won&#39;t be lost because postgres_data volume will take care of that.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;ports: - &quot;5433:5432&quot;&lt;/code&gt;  and here I&#39;m mapping host port 5433 to container port 5432. 5433 Because I already have PostgreSQL running on my computer on port 5432. Using 5433 avoids conflicts. But inside the Docker network, other containers will still connect to port 5432.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;volumes: - postgres_data:/var/lib/postgresql/data&lt;/code&gt; as we talked about it above in env_file this one uses a named volume instead of a bind mount. This will persists my database data even when I stop or restart the containers which I most likely will. If I run &lt;code&gt;docker-compose down&lt;/code&gt;, the containers are removed but the &lt;code&gt;postgres_data&lt;/code&gt; volume remains with all my data. Only &lt;code&gt;docker-compose down -v&lt;/code&gt; (with the &lt;code&gt;-v&lt;/code&gt; flag) deletes volumes.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;healthcheck&lt;/code&gt; runs &lt;code&gt;pg_isready&lt;/code&gt; every 5 seconds to check if PostgreSQL is actually ready to accept connections.&lt;/p&gt;
&lt;h3 id=&quot;running-everything&quot;&gt;Running Everything&lt;/h3&gt;
&lt;p&gt;now I have this in place I can start my application with one command.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;docker&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; compose up&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This command will build the web image (if it doesn&#39;t exist), pulls the postgres image, creates containers, sets up networking, and starts everything.&lt;/p&gt;
&lt;p&gt;If I want to have it run in the background:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;docker&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; compose up -d&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;then I  keep developing the application
I can check logs with:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;docker&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; compose logs -f web&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;docker&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; compose logs -f db&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;When I&#39;m done:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;docker&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; compose down&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This  stops and removes containers, but keeps volumes (my database data). If I want a completely fresh start:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;docker&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; compose down -v&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;-v&lt;/code&gt; flag deletes volumes too. Be careful with this because it will actually wipe your database!&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;database migrations&lt;/strong&gt;:&lt;/p&gt;
&lt;p&gt;initializing alembic&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;docker compose exec web uv run alembic init alembic&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;auto generating the migration scripts&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;docker compose exec web uv run alembic --autogenerate -m&amp;#39;add is_superuser in users table&amp;#39;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;upgrading the database:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;docker compose exec web uv run alembic upgrade head&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Everything is the same as using Alembic normally, except we preface it with &lt;code&gt;docker compose exec web&lt;/code&gt; because we need to run the command inside the container where our code and virtual environment live. By using exec web, I am telling Docker: &quot;Go inside the running container named &#39;web&#39; and run this command there.&quot; Since the web container is already on the same network as the database, Alembic will have no trouble finding it.
you can go into the shell in container as well with &lt;code&gt;docker compose exec -it web bash&lt;/code&gt; to see how things are looking.&lt;/p&gt;
&lt;p&gt;Overall, you can notice  that we only used just one compose.yml file, and it works fine for now. But Docker Compose is powerful. We can go even further by using overrides. like we can create a compose.override.yml file and overrides the services we have inside compose.yaml. Think about how handy this is when you have different environments like development, staging, and production. but it doesn&#39;t have to be an application with all those environments. you could even have a smaller application like this one and do that. in fact let&#39;s create compose.override.yml file, in this file I am going to take instructions from compose.yml file we already have and instead put them in this override. this override will serve as our development compose file with things like volume mounting. we will take out the volumes, development command as well the environment instructions. Our compose.override.yml will have:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;yaml&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;services&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;  web&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    environment&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; POSTGRES_SERVER=db&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; POSTGRES_PORT=5432&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    volumes&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; ./app:/web-backend/app&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;      -&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; ./alembic:/web-backend/alembic&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;    command&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;:&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; uv run fastapi dev --host 0.0.0.0 --port 8000&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;then we will remain with compose.yml like this:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;services:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  web:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    build: .&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    ports:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      - &amp;quot;8000:8000&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    env_file:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      - .env&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    depends_on:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      db:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        condition: service_healthy&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        restart: true&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  db:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    image: postgres:17&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    env_file:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      - .env&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    ports:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      - &amp;quot;5433:5432&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    volumes:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      - postgres_data:/var/lib/postgresql/data&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    healthcheck:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      test: [&amp;quot;CMD-SHELL&amp;quot;, &amp;quot;pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}&amp;quot;]&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      interval: 10s&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      timeout: 5s&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      retries: 5&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;volumes:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  postgres_data:&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;now, I can still just run:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;docker compose up&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Docker Compose automatically looks for the base file and then the override file, merging them together. Everything runs exactly as it did before! If you want to use different names, you can specify them manually with the -f flag (like docker compose -f compose.yml -f compose.prod.yml up).&lt;/p&gt;
&lt;h3 id=&quot;wrapping-up&quot;&gt;wrapping up&lt;/h3&gt;
&lt;p&gt;I’ve found Docker containers to be really convenient for both developing and deploying applications. But, you need to have your configurations down. and the more you use it, the more you get comfortable with the commands and what your particular application needs.
I know I’ve only touched on some of these things lightly and skipped others, but these were the essentials I needed to get this specific setup running.&lt;/p&gt;
&lt;p&gt;Also, Happy new Year!&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>SQLAlchemy Through Its Core API</title>
        <published>2025-12-20T00:00:00+00:00</published>
        <updated>2025-12-20T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/sqlalchemy-through-its-core-api-engine-metadata-table/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/sqlalchemy-through-its-core-api-engine-metadata-table/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/sqlalchemy-through-its-core-api-engine-metadata-table/">&lt;p&gt;When working with databases from a backend application, most of us default to using an ORM (Object Relational Mapper) to avoid writing raw SQL and dealing with lower-level database details. In Python, SQLAlchemy is one of the most commonly used tools for this.&lt;/p&gt;
&lt;p&gt;What’s interesting about SQLAlchemy is that it is not just an ORM. It consists of two main layers: &lt;strong&gt;Core&lt;/strong&gt; and &lt;strong&gt;ORM&lt;/strong&gt;, where the ORM is built on top of Core. In this article, I’ll focus on working with SQLAlchemy through its Core API rather than the ORM.
Why? Well, while I do like the ORM side in terms of how it feels more Python than database, I think it has more magic to work with than the Core API. The things you can do with the ORM, you can also do with Core, so this article is for anyone who is interested in learning about the Core API. Presumably, some familiarity with the ORM API helps, but it’s not required to go through the article. As for which one to use, you can compare them yourself and decide what you like.&lt;/p&gt;
&lt;p&gt;This article assumes a basic understanding of Python programming language and some familiarity with interacting with relational databases from a Python application.&lt;/p&gt;
&lt;h3 id=&quot;what-is-sqlalchemy&quot;&gt;What is SQLAlchemy?&lt;/h3&gt;
&lt;p&gt;SQLAlchemy is a Python library for working with relational databases. It allows you to define Python objects that represent SQL concepts such as tables, columns, and queries, which are then compiled into actual SQL statements that your database can execute.&lt;/p&gt;
&lt;p&gt;sqlalchemy has two components: core and orm. core as the foundation, the database toolkit. This is where engines, connections, metadata, tables, and sql expressions are. with core, you explicitly build sql using python objects, for example:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;user_table.insert().values(name=&amp;quot;john&amp;quot;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;the line above is a description of an insert sql statement. when you actually execute a statement, core compiles it into a sql string using a database-specific dialect (a layer that translates SQLAlchemy’s generic instructions into the exact SQL and driver calls a specific database understands), and then hands it off to the dbapi (psycopg2, mysqlclient, etc), which is what actually talks to the database. The ORM sits on top of Core. It adds two main things: object mapping (which are Python classes but mapped as database tables, and table rows which becomes Python objects) and sessions (which manage transactions and track changes to those objects).&lt;/p&gt;
&lt;h3 id=&quot;installation&quot;&gt;installation&lt;/h3&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;pip install SQLAlchemy&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;or with uv:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;uv add SQLAlchemy&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;h3 id=&quot;the-engine&quot;&gt;The engine&lt;/h3&gt;
&lt;p&gt;The engine is SQLAlchemy&#39;s way of managing connections to your database. in most cases, you don&#39;t want to create a new connection every single time you need to run a query. Opening a database
connection involves different other things under the hood like TCP handshakes, authentication, and session setup that can take up a little bit of time.Instead, you would open a few connections, keep them alive, and reuse them. This is what they call connection pooling.&lt;/p&gt;
&lt;p&gt;SQLAlchemy&#39;s engine does this automatically. When you create an engine, it sets up a connection pool (by default, a QueuePool with 5 slots). and with sqlalchemy it doesn&#39;teven  actually connect to the database yet! instead the engine is lazy so they call!,  it only creates connections when you actually need them.
The first time you try to execute something (use the engine to connect engine.connect, create tables, run a query, whatever), the engine grabs a connection from the pool. If the pool is empty, it creates a new connection. When your query is done ,  the connection goes back into the pool for reuse instead of being closed.&lt;/p&gt;
&lt;p&gt;You create an engine like this:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; sqlalchemy&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; create_engine&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;engine&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; create_engine&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;    &amp;quot;postgresql+psycopg2://john:super_secret@localhost:5432/test_db&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The URL above contains:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;postgresql&lt;/code&gt; - the database dialect (can be PostgreSQL, MySQL, SQLite, etc.)&lt;/li&gt;
&lt;li&gt;&lt;code&gt;psycopg2&lt;/code&gt; - the DBAPI driver (the Python library that actually talks to PostgreSQL)&lt;/li&gt;
&lt;li&gt;&lt;code&gt;john:super_secret&lt;/code&gt; - username and password&lt;/li&gt;
&lt;li&gt;&lt;code&gt;localhost:5432&lt;/code&gt; - host and port&lt;/li&gt;
&lt;li&gt;&lt;code&gt;test_db&lt;/code&gt; - database name&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;You typically create one engine per database when your application starts up and keep it around for the lifetime of your application. The pool handles everything else - growing when you need more connections, recycling stale ones, limiting how many can be active at once.&lt;/p&gt;
&lt;p&gt;depending on you application needs you may need to tune this pool say may be your application is handling lots of concurrent requests, you can pass parameters like:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;engine&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; create_engine&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;    &amp;quot;postgresql+psycopg2://john:super_secret@localhost:5432/test_db&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;    pool_size&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;10&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;        # keep 10 connections in the pool&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;    max_overflow&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;5&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;      # allow 5 extra connections beyond pool_size if needed&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt;    pool_recycle&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;3600&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;   # recycle connections after 1 hour&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;keeping the defaults is usually fine for most applications.&lt;/p&gt;
&lt;h3 id=&quot;the-metadata&quot;&gt;The MetaData&lt;/h3&gt;
&lt;p&gt;MetaData is where you register all your table definitions. It is a collection registry that stores Table objects, keyed by their table name.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why do you need it? A few reasons:&lt;/strong&gt;
When you define tables with foreign keys, SQLAlchemy needs to know about all the related tables to properly set up those relationships. The MetaData object is where it will look them up.
When you actually create the tables (we&#39;ll get to this), SQLAlchemy needs to create them in the right order - tables without foreign keys first, then the tables that reference them. MetaData tracks all the tables to figure out this dependency order.
And the other one is that it acts as a central place to keep all your schema definitions together. This is very important when you want to use a schema migration tool like Alembic (you just specify this metadata, because it essentially holds all the tables in your entire application).&lt;/p&gt;
&lt;p&gt;You create it like this:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; sqlalchemy&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; MetaData&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;metadata&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; MetaData&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That&#39;s it. Now you have an empty registry ready to hold table definitions.&lt;/p&gt;
&lt;p&gt;You typically create one MetaData object at the module level (like in a &lt;code&gt;models.py&lt;/code&gt; or &lt;code&gt;schema.py&lt;/code&gt; file) and use it throughout your application. When you define tables, you&#39;ll pass this metadata object to them, and they&#39;ll register themselves automatically.&lt;/p&gt;
&lt;p&gt;In case you get curious and want to see, it has more attributes you can inspect yourself. One of them is the .tables attribute which is a dictionary of all registered tables:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;metadata&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;tables&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # empty dict at first&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;metadata&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;tables&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;keys&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;())&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # will show table names after you define them&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If you&#39;re using the ORM, you don&#39;t see this MetaData object explicitly - it&#39;s created for you when you make your &lt;code&gt;Base = declarative_base()&lt;/code&gt;. But it&#39;s still there, and you can access it with &lt;code&gt;Base.metadata&lt;/code&gt;.&lt;/p&gt;
&lt;h3 id=&quot;defining-tables&quot;&gt;Defining Tables&lt;/h3&gt;
&lt;p&gt;To add tables, you use the Table constructor:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from sqlalchemy import Table, Column, Integer, String, ForeignKey&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;user_table = Table(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    &amp;quot;users&amp;quot;, # tablename(as a string), this is what the table will be called in the database&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    metadata, # The MetaData object to register with&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # Column definitions&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    Column(&amp;quot;id&amp;quot;, Integer, primary_key=True),&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    Column(&amp;quot;name&amp;quot;, String(25), nullable=False),&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    Column(&amp;quot;email&amp;quot;, String(255), unique=True, nullable=False),&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;address_table = Table(&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    &amp;quot;addresses&amp;quot;,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    metadata,&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    Column(&amp;quot;id&amp;quot;, Integer, primary_key=True),&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    Column(&amp;quot;user_id&amp;quot;, ForeignKey(&amp;quot;users.id&amp;quot;), nullable=False),&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    Column(&amp;quot;street&amp;quot;, String(100)),&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    Column(&amp;quot;city&amp;quot;, String(50)),&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Notice that user_table and address_table are just  Python variable name!. The actual table name in the database is &quot;users&quot; and &quot;addresses&quot;.
When you define a table with a foreign key like ForeignKey(&quot;users.id&quot;), you&#39;re referencing the database table name (&quot;users&quot;), not the Python variable name (user_table).
After defining these tables, they&#39;re automatically registered in the metadata:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;print(metadata.tables.keys())&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;# dict_keys([&amp;#39;users&amp;#39;, &amp;#39;addresses&amp;#39;])&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;strong&gt;Creating and Dropping Tables&lt;/strong&gt;
Once you&#39;ve defined your tables, you need to actually create them in the database. You can do this with:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;metadata.create_all(engine)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This connects to the database and issues CREATE TABLE statements for all tables in the metadata. SQLAlchemy does this in a clever way, it will create tables in dependency order (tables without foreign keys first), and it checks if tables already exist before trying to create them.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;metadata.drop_all(engine)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This issues DROP TABLE statements for all tables, in reverse dependency order (tables with foreign keys get dropped first).
You can also create or drop individual tables:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;user_table.create(engine)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;user_table.drop(engine)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now, In real applications, you rarely use create_all() or drop_all() directly. Instead, you would use a schema migration tool like Alembic to manage schema changes over time. We&#39;ll touch on this next.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Working with Columns&lt;/strong&gt;
After you&#39;ve defined a table, you can reference its columns using the .columns attribute or you can just use .c:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;# Accessing columns&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;&amp;gt;&amp;gt;&amp;gt; user_table.c.name&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Column(&amp;#39;name&amp;#39;, String(length=25), table=&amp;lt;users&amp;gt;, nullable=False)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;&amp;gt;&amp;gt;&amp;gt; user_table.c.email&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Column(&amp;#39;email&amp;#39;, String(length=255), table=&amp;lt;users&amp;gt;, nullable=False)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;# Get all column names&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;print(user_table.c.keys())  # [&amp;#39;id&amp;#39;, &amp;#39;name&amp;#39;, &amp;#39;email&amp;#39;]&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;referencing columns when building queries:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from sqlalchemy import select&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;# Build a SELECT query&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;stmt = select(user_table).where(user_table.c.name == &amp;quot;john&amp;quot;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;# Execute it&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;with engine.connect() as conn:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    result = conn.execute(stmt)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    for row in result:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        print(row)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The with statement here is a context manager. When you do with engine.connect() as conn, it grabs a connection from the pool and assigns it to conn. When the block exits (either normally or because of an error), the connection automatically gets returned to the pool. You don&#39;t have to manually close it. This is something you&#39;ll use most of the time when working with connections.&lt;/p&gt;
&lt;p&gt;now, the example above uses engine.connect(), which is called &quot;commit as you go&quot; - you need to explicitly commit any changes:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from sqlalchemy import insert&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;with engine.connect() as conn:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    stmt = insert(user_table).values(name=&amp;quot;john&amp;quot;, email=&amp;quot;john@example.com&amp;quot;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    conn.execute(stmt)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    conn.commit()  # You have to call commit explicitly&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There&#39;s another way using engine.begin(), which automatically commits when the block exits (or rolls back if there&#39;s an error):&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;with engine.begin() as conn:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    stmt = insert(user_table).values(name=&amp;quot;john&amp;quot;, email=&amp;quot;john@example.com&amp;quot;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    conn.execute(stmt)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # Automatically commits here when the block exits&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;h3 id=&quot;using-metadata-with-alembic&quot;&gt;Using MetaData with Alembic&lt;/h3&gt;
&lt;p&gt;When you use Alembic for database schema migrations, you need to tell it about your MetaData object. In your alembic/env.py file, you&#39;ll do something like this:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; myapp&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;models&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; metadata&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt; #or whereever you created your metadata object&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;h3 id=&quot;point-alembic-to-your-metadata&quot;&gt;Point Alembic to your metadata&lt;/h3&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;target_metadata = metadata&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If you&#39;re using the ORM, it&#39;s the same idea, just with Base.metadata:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; myapp&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;models&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; Base&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;target_metadata&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span&gt; Base&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;metadata&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The MetaData object is what Alembic uses to understand your schema. This is why having all your tables registered in one MetaData (or one ORM Base) is so important.
I actually talked a little but about this in another  &lt;a rel=&quot;external&quot; href=&quot;https://www.niyonshutiemmanuel.com/blog/database-schema-migrations-with-alembic&quot;&gt;blog&lt;/a&gt; if you are interested.&lt;/p&gt;
&lt;h3 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h3&gt;
&lt;p&gt;Every application using SQLAlchemy needs an engine to manage database connections. Whether you use Core or ORM, you always start with an engine. The MetaData object is the registry for your table schemas. In Core, you create the metadata instance yourself. In ORM, it&#39;s created for you via Base.metadata.&lt;/p&gt;
&lt;p&gt;As I said starting the article,  the choice between Core and ORM depends on your preference and how you want your application to be.  but, I think starting with Core and getting comfortable with it before moving to ORM can  help you avoid the magic you might feel from time to time when you jump straight into using the ORM. With that said there is nothing wrong with jumping straight to using the orm api. Both are valid and even better, you can mix both in the same application.&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>Environment Configurations Using Pydantic Settings</title>
        <published>2025-12-15T00:00:00+00:00</published>
        <updated>2025-12-15T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/loading-and-dealing-with-environment-configurations-using-pydantic-settings/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/loading-and-dealing-with-environment-configurations-using-pydantic-settings/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/loading-and-dealing-with-environment-configurations-using-pydantic-settings/">&lt;p&gt;Pydantic v2 moved the functionality for handling application settings and environment variables into a separate package called &lt;code&gt;pydantic_settings&lt;/code&gt;. It provides an elegant and reliable way to manage your application&#39;s configuration.
If you haven&#39;t used pydantic_settings, you might be familiar with other packages like python-dotenv for loading environment variables, or some other package to achieve the same goal. The advantage of pydantic_settings is that it not only loads your environment variables but also brings all of Pydantic&#39;s powerful validation and type-checking features.
In this article, I&#39;m going to show you how you can use pydantic_settings to load your environment variables and also how you can easily switch between different environments. It&#39;s one easy technique I&#39;ve found to be simple and useful for switching different environments, but there are probably other ways you can do it, so don&#39;t take it as the only way.
This article assumes some familiarity with the Python programming language, and while not strictly required, it&#39;s better to also have some basic understanding of working with Pydantic.&lt;/p&gt;
&lt;h3 id=&quot;installation&quot;&gt;Installation&lt;/h3&gt;
&lt;p&gt;You&#39;ll need to install the pydantic_settings package using your preferred package manager. I recommend creating a virtual environment first so the package stays project-specific rather than installed system-wide.&lt;/p&gt;
&lt;p&gt;Installing with pip:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;pip install pydantic_settings&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;or with uv&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;uv add pydantic_settings&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;h3 id=&quot;basesettings&quot;&gt;BaseSettings&lt;/h3&gt;
&lt;p&gt;Normally, when you use Pydantic in a  project, your models inherit from pydantic.BaseModel. However, depending on your use case, you might inherit from other base classes. for example, if you&#39;re building a backend with SQLModel in a FastAPI project, your models would inherit from SQLModel instead.
For application settings and environment variables specifically, your Pydantic model(s) inherit from a different base class: pydantic_settings.BaseSettings.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from pydantic_settings import BaseSettings&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class Settings(BaseSettings):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;          database_url: str&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Settings is just an arbitrary name, you can call it whatever makes sense for your project. I&#39;m using database_url as an example environment variable here.
In this code, I&#39;ve imported the BaseSettings class, created a Settings class, and added one required field. With this setup, BaseSettings assumes that database_url is already loaded into the system environment when you run your script or start your application.&lt;/p&gt;
&lt;p&gt;There are several ways you can load your application settings, but pydantic_settings offers a particularly elegant approach to handle this.&lt;/p&gt;
&lt;h3 id=&quot;how-does-pydantic-settings-reads-these-values&quot;&gt;How does pydantic settings reads these values?&lt;/h3&gt;
&lt;p&gt;By default, pydantic settings (&lt;code&gt;BaseSettings&lt;/code&gt;) tends to read these values in order of priority.
it will read arguments passed directly when instantiating the class , then values from the environment variables already present in the shell or os environment, and then the default values specified in the class definition.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Loading Environment Variables with load_dotenv&lt;/strong&gt;
You could load your environment variables using load_dotenv at the top of your file. For example, assuming you have database_url defined in your .env file:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from dotenv import load_dotenv&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from pydantic_settings import BaseSettings&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;load_dotenv()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class Settings(BaseSettings):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    database_url: str&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  &lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;s = Settings()&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This approach works perfectly fine. The load_dotenv() function reads your .env file and loads all variables into the system environment before you instantiate Settings.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Specifying the Environment File at Instantiation&lt;/strong&gt;
Alternatively, you can specify which environment file to use when creating an instance:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;s = Settings(_env_file=&amp;quot;.env&amp;quot;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;With this Pydantic Settings will load variables from the specified file. The &lt;code&gt;_env_file&lt;/code&gt; parameter overrides any env_file setting in model_config (which we haven&#39;t set in our example yet).
Now, you might think, &quot;Well, I can just load a different environment file say .env.prod using &lt;code&gt;_env_file&lt;/code&gt; and it will use that instead of .env.&quot; However, it won&#39;t work as you&#39;d expect. Your settings will still be those from .env that were loaded by load_dotenv(). If you&#39;ve already used load_dotenv(), the&lt;code&gt; _env_file&lt;/code&gt; parameter won&#39;t override those loaded variables. The load_dotenv() approach loads variables into the system environment immediately (before your class is even instantiated), while&lt;code&gt; _env_file&lt;/code&gt; is handled by Pydantic Settings itself during instantiation.&lt;/p&gt;
&lt;h3 id=&quot;working-with-multiple-environments&quot;&gt;Working with Multiple Environments&lt;/h3&gt;
&lt;p&gt;The other way, which may be simpler in this situation, is that you can use &lt;code&gt;SettingsConfigDict&lt;/code&gt; that Pydantic settings provides. You can keep your .env file and prefix your environment variables. For example:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;.env&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;DEV_DATABASE_URL=postgresql+psycopg2://username:password@hostname:5432/my_app_dev_db&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;PROD_DATABASE_URL=postgresql+psycopg2://username:password@hostname:5432/my_app_prod_db&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;then in And in your Pydantic model configurations&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from pydantic_settings import BaseSettings, SettingsConfigDict&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class Settings(BaseSettings):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    database_url: str&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt; &lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class ProdConfig(Settings):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    model_config = SettingsConfigDict(env_prefix=&amp;quot;PROD_&amp;quot;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;When you want to use the &lt;code&gt;ProdConfig&lt;/code&gt;, Pydantic settings will take care of this by looking for variables prefixed with the prefix you just specified. Now you can go from there and add more classes and environments.&lt;/p&gt;
&lt;p&gt;Normally, you rarely need to explicitly or I might say use the load_dotenv. Instead, this &lt;code&gt;SettingsConfigDict&lt;/code&gt; and &lt;code&gt;BaseSettings&lt;/code&gt; will take care of the loading of environment variables for you. As you see, we used the load_dotenv but we didn&#39;t install python-dotenv as we would normally do! pydantic_settings comes with it.
Taking a look at my installation here:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Resolved 8 packages in 1.03s&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Installed 7 packages in 47ms&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt; + annotated-types==0.7.0&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt; + pydantic==2.12.5&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt; + pydantic-core==2.41.5&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt; + pydantic-settings==2.12.0&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt; + python-dotenv==1.2.1   &lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt; + typing-extensions==4.15.0&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt; + typing-inspection==0.4.2&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You can see the package installed right there as well.
So you can specify your env file inside the model config, and Pydantic will load it:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from pydantic_settings import BaseSettings, SettingsConfigDict&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class Settings(BaseSettings):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    model_config = SettingsConfigDict(env_file=&amp;quot;.env&amp;quot;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    database_url: str&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt; &lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class ProdConfig(Settings):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    model_config = SettingsConfigDict(env_prefix=&amp;quot;PROD_&amp;quot;)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;With this, it will take care of loading your environment variables, prefix as the same, and do the validations as well.
You might want to set &lt;code&gt;extra=&quot;ignore&quot;&lt;/code&gt;when you want Pydantic to ignore extra variables that you might have in your environment variables. You can also set the encoding you want your env file to be read in, as it defaults to your operating system&#39;s encoding. There are more important arguments you can specify!
So you can override your model_config at instantiation time. you might not even need to have the separate ProdConfig class anyways. You can keep everything inside the settings and then specify the env file you want to load from at instantiation time.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from pydantic_settings import BaseSettings, SettingsConfigDict&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class Settings(BaseSettings):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    model_config = SettingsConfigDict(env_file=&amp;quot;.env&amp;quot;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    database_url: str&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;So, you can go from there as well.&lt;/p&gt;
&lt;h3 id=&quot;switching-environments&quot;&gt;Switching environments&lt;/h3&gt;
&lt;p&gt;This is one of many ways you can work with your application environment variables, but what I want to talk about is using separate config Pydantic models, inheriting from this base settings, keeping everything inside one env file, prefixing them accordingly, having an environment state variable to determine which environment to use, and overriding it whenever you want to switch which environment you want your application to run in.&lt;/p&gt;
&lt;p&gt;Consider you have your application in this folder structure:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;├── app&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;│   ├── __init__.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;│   ├── config.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;│   └── main.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;├── tests&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;│   ├── __init__.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;│   ├── conftest.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;│   └── test_main.py&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;└── .env&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You can have something like this in your config:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from functools import lru_cache&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;from pydantic_settings import BaseSettings, SettingsConfigDict&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class BaseConfig(BaseSettings):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    ENV_STATE: str&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    model_config = SettingsConfigDict(env_file=&amp;quot;.env&amp;quot;, extra=&amp;quot;ignore&amp;quot;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class GlobalConfig(BaseConfig):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    DATABASE_URL: str&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class DevConfig(GlobalConfig):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    model_config = SettingsConfigDict(env_prefix=&amp;quot;DEV_&amp;quot;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class TestConfig(GlobalConfig):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    model_config = SettingsConfigDict(env_prefix=&amp;quot;TEST_&amp;quot;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;class ProdConfig(GlobalConfig):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    model_config = SettingsConfigDict(env_prefix=&amp;quot;PROD_&amp;quot;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;@lru_cache()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;def get_config(env_state: str):&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    config_dict = {&amp;quot;dev&amp;quot;: DevConfig, &amp;quot;test&amp;quot;: TestConfig, &amp;quot;prod&amp;quot;: ProdConfig}&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    return config_dict[env_state]()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;config = get_config(BaseConfig().ENV_STATE)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now, what is happening in here? It&#39;s still the same concept, but let me explain.&lt;/p&gt;
&lt;p&gt;We have a &lt;code&gt;BaseConfig&lt;/code&gt; that reads the &lt;code&gt;ENV_STATE&lt;/code&gt; environment variable. This is the variable that determines which environment we&#39;re running in. It also sets up the basic configuration like pointing to our &lt;code&gt;.env &lt;/code&gt;file and telling Pydantic to ignore extra variables that might be in the environment but not in our model.
Then we have GlobalConfig, which inherits from BaseConfig. This is where we define fields that might be common across all environments. In this case, we have &lt;code&gt;DATABASE_URL&lt;/code&gt;. Then we have three environment-specific configs: &lt;code&gt;DevConfig&lt;/code&gt;, &lt;code&gt;TestConfig&lt;/code&gt;, and &lt;code&gt;ProdConfig&lt;/code&gt;. Each one inherits from &lt;code&gt;GlobalConfig&lt;/code&gt; and sets an &lt;code&gt;env_prefix&lt;/code&gt; in &lt;code&gt;model_config&lt;/code&gt;. So when we use &lt;code&gt;DevConfig&lt;/code&gt;, Pydantic will look for variables like &lt;code&gt;DEV_DATABASE_URL&lt;/code&gt; in our environment(the &lt;code&gt;.env&lt;/code&gt;file since thats what we specified). When we use &lt;code&gt;ProdConfig&lt;/code&gt;, it will look for &lt;code&gt;PROD_DATABASE_URL&lt;/code&gt;, and so on.
The &lt;code&gt;get_config&lt;/code&gt; function is where things get a little bit interesting. It&#39;s decorated with &lt;code&gt;@lru_cache()&lt;/code&gt;, which means once we call it with a specific &lt;code&gt;env_state&lt;/code&gt;, it will cache  the result. I like to put it this way because I don&#39;t need to keep re-instantiating or creating the config object every time I import it in the application since it would also read the &lt;code&gt;.env&lt;/code&gt; file as well and I don&#39;t change these environment variables that often so I keep it this way in most cases.
The &lt;code&gt;get_config&lt;/code&gt; function takes an &lt;code&gt;env_state&lt;/code&gt; string and returns the appropriate config class instance. The &lt;code&gt;config_dict&lt;/code&gt; maps environment names to their corresponding config classes.  If &lt;code&gt;env_state&lt;/code&gt; is &lt;code&gt;&quot;dev&quot;&lt;/code&gt;, it returns &lt;code&gt;DevConfig()&lt;/code&gt;, if it&#39;s &lt;code&gt;&quot;prod&quot;&lt;/code&gt;, it returns&lt;code&gt; ProdConfig()&lt;/code&gt;, and so on.&lt;/p&gt;
&lt;p&gt;Finally, at the bottom, we have &lt;code&gt;config = get_config(BaseConfig().ENV_STATE)&lt;/code&gt;. This is where we actually create our config instance. We instantiate &lt;code&gt;BaseConfig&lt;/code&gt; just to read the &lt;code&gt;ENV_STATE&lt;/code&gt; variable, then pass that to &lt;code&gt;get_config&lt;/code&gt;, which returns the appropriate environment config.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;.env &lt;/code&gt;file will look something like this:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;ENV_STATE=dev&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;#delopment environment&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;DEV_DATABASE_URL=postgresql://localhost/dev_db&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;# Test environment&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TEST_DATABASE_URL=postgresql://localhost/test_db&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;# Production environment&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;PROD_DATABASE_URL=postgresql://production_server/prod_db&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;now, all your configurations live in one place, and switching environments is as simple as changing the &lt;code&gt;ENV_STATE &lt;/code&gt;variable. If you are writting tests for your application for example with pytest. You can switch the environment at the top of your &lt;code&gt;conftest&lt;/code&gt; by just:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;import os&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;os.environ[&amp;quot;ENV_STATE&amp;quot;] = &amp;quot;test&amp;quot;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As your application grows and you add more configuration options, you just add them to the appropriate &lt;code&gt;config class&lt;/code&gt; with the right prefix in your &lt;code&gt;.env&lt;/code&gt;file.
This is the way I&#39;ve been using this, and it&#39;s served me well so far. There are many ways you can achieve the
same thing and you can also enhance this one to suit your use case. I hope you learned something from this!&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>http and websockets</title>
        <published>2025-12-12T00:00:00+00:00</published>
        <updated>2025-12-12T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/websockets-and-http/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/websockets-and-http/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/websockets-and-http/">&lt;p&gt;HTTP is a communication protocol, but what does that actually mean?
The clean way to think about it is that HTTP is simply a set of rules that different programs agree to follow. If two programs follow the same rules, they can communicate.
This also means something subtle but important: HTTP only exists as behavior implemented in software. There is no single &quot;HTTP program&quot; you install. Instead, many different applications implement the protocol: web browsers implement HTTP clients, servers like Nginx, Apache, Flask, and FastAPI implement HTTP servers, proxies and tools implement their own HTTP parsing, and so on. All of them follow the same specification, and that shared behavior is what makes HTTP.
In the context of understanding WebSockets, the most important thing to know about HTTP is that it is strictly request-response. The client has to send a request first so the server can respond. And once the response is complete, the exchange is considered finished even if the underlying TCP connection (the transport layer that actually delivers the bytes; more on that later, maybe!) remains open for reuse. In this model, the server cannot spontaneously send an unrequested HTTP message to a client. If you think about it, this makes real-time communication difficult.
If you want your application to support notifications, live updates, chats, or anything where the server needs to push data to the client, HTTP becomes limiting. People still achieve this goal through polling repeatedly asking the server if there are new messages or information the client wants. But this is not so reliable and http was not designed for that kind of things. RFC 6455 refers to it as &quot;an abuse of HTTP.&quot; You can &lt;a rel=&quot;external&quot; href=&quot;https://www.rfc-editor.org/rfc/rfc6455#page-37&quot;&gt;read&lt;/a&gt; more if you&#39;re interested.&lt;/p&gt;
&lt;p&gt;So, &lt;strong&gt;how do WebSockets work?&lt;/strong&gt; Well, a WebSocket connection starts as a normal HTTP GET request with special Upgrade and Connection headers that check if the server supports the WebSocket protocol. If it does, they switch protocols and abandon the request-response cycle of HTTP.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The Handshake&lt;/strong&gt;
When the browser initiates a WebSocket connection, it sends what looks like a normal HTTP request:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;GET /chat HTTP/1.1&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Host: localhost:8000&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Connection: Upgrade&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Pragma: no-cache&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Cache-Control: no-cache&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;User-Agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Mobile Safari/537.36&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Upgrade: websocket&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Origin: http://127.0.0.1:8000&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Sec-WebSocket-Version: 13&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Accept-Encoding: gzip, deflate, br, zstd&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Accept-Language: en-GB,en-US;q=0.9,en;q=0.8,zh-TW;q=0.7,zh;q=0.6,fr-FR;q=0.5,fr;q=0.4,en-CA;q=0.3,et;q=0.2,zh-CN;q=0.1,rw;q=0.1&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Sec-WebSocket-Key: P/EAZNx+WFeGI/yCqKhrBQ==&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You can recognize some unusual headers in this request,  Upgrade: websocket header is the client asking for switching the protocol, The Sec-WebSocket-Key  header is a random base64-encoded value. The server takes this key, combines it with a magic GUID, hashes it, and sends back the result. If the response matches what the client expects, the server understands WebSocket.&lt;/p&gt;
&lt;p&gt;If the server accepts, it responds with something like this:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;HTTP/1.1 101 Switching Protocols&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Upgrade: websocket&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Connection: Upgrade&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Sec-WebSocket-Accept: K2pIvvYi4t4w7CVlgwwREj5ra28=&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Sec-WebSocket-Extensions: permessage-deflate&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;date: Sat, 13 Dec 2025 18:41:19 GMT&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;server: uvicorn&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This HTTP 101 response is the last HTTP message you&#39;ll ever see on this connection. After this, the protocol changes. this is interesting! The connection now will be using websocket pritocol.
now it will be using binary frames Instead of text-based HTTP messages with headers, data is now sent in frames(small binary structures with):
An opcode (the type of frame it is?)
A length field
The actual payload data&lt;/p&gt;
&lt;p&gt;There are different frame types as well and I am not going into that now, the key difference is that no HTTP headers on every message. basically, when you send message like &quot;Hello&quot;, you&#39;re sending ~7 bytes total, not 200+ bytes of HTTP overhead. This is one of the reasons why WebSockets are efficient for real-time communication.
&lt;strong&gt;How does it stay alive then?&lt;/strong&gt;
How do you know if the other side is still there? TCP doesn&#39;t tell you if the connection is dead until you try to send something! WebSockets solve this with Ping and Pong frames.
Either side can send a Ping frame at any time. When you receive a Ping, you must respond with a Pong frame containing the same data. and this will prevent intermediaries (proxies, load balancers) from closing idle connections and Verify the remote endpoint is still responsive.&lt;/p&gt;
&lt;p&gt;and just like there&#39;s a handshake to open the connection, there&#39;s one to close it. Either side can initiate by sending a Close frame (opcode 0x8), which can optionally include:&lt;/p&gt;
&lt;p&gt;A status code (like 1000 for &quot;normal closure&quot; or 1001 for &quot;going away&quot;)
A reason string (for debugging)&lt;/p&gt;
&lt;p&gt;When you receive a Close frame, you must respond with your own Close frame, then close the underlying TCP connection. This two-way close handshake ensures neither side loses data, both parties agree the conversation is over before the TCP connection drops.&lt;/p&gt;
&lt;p&gt;some common status codes:
1000: Normal closure (we&#39;re done)
1001: Going away (server shutting down, browser navigating away)
1002: Protocol error (malformed frame received)
1006: Abnormal closure (connection lost without Close frame)&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>Database schema migrations with Alembic</title>
        <published>2025-12-07T00:00:00+00:00</published>
        <updated>2025-12-07T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/database-schema-migrations-with-alembic/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/database-schema-migrations-with-alembic/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/database-schema-migrations-with-alembic/">&lt;p&gt;This article is about keeping your database schema in sync with your code. It is not about moving data between databases, which is a different problem that requires different tools.&lt;/p&gt;
&lt;p&gt;People often refer to this process as “database migration,” but what we’re really talking about here is the ongoing work of evolving your database structure as your application changes.&lt;/p&gt;
&lt;p&gt;If you’re not familiar with database schema migration, I will give you an example: imagine your application needs to store some data(users, items, or other information). You define tables for this data in your code, often using a tool like an ORM (object relational mapper for example, SQLAlchemy that lets you describe the database structure as Python objects). Or, you might use lower-level tools to connect your code directly to the database. Either way, your code now has a blueprint of the tables you want in your database.&lt;/p&gt;
&lt;p&gt;A migration tool ensures that your database matches that blueprint. Later, if you need to modify a table say, remove a column, rename a field, or change a datatype the migration tool applies those changes automatically. For example, if your users table initially has a username column but you later decide it’s unnecessary, the tool will remove it from the database as well. This is exactly what a tool like Alembic were made for.&lt;/p&gt;
&lt;p&gt;This article assumes some familiarity with Python, but the concepts of database schema migration apply to any programming language. Here, we focus on Alembic as the example tool.&lt;/p&gt;
&lt;h2 id=&quot;overview-of-how-it-works&quot;&gt;Overview of How it works&lt;/h2&gt;
&lt;p&gt;Alembic tracks schema changes as Python files called revisions. Each revision is a script containing two functions:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;-&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; upgrade&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # applies the change to your database&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;-&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; downgrade&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # undoes it (in case you care about rollbacks)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These revision files live in a versions/ directory and are linked together using a down_revision attribute. The first migration sets down_revision = None, and each subsequent migration points to its predecessor. This is how Alembic determines the correct order to apply revisions.&lt;/p&gt;
&lt;h2 id=&quot;the-migration-environment&quot;&gt;The migration environment&lt;/h2&gt;
&lt;p&gt;Before you can create any revisions, you need to set up what Alembic calls the &quot;migration environment.&quot; This is the directory structure where migration scripts and configuration live. You create it once and maintain it alongside your code.&lt;/p&gt;
&lt;h2 id=&quot;installing-initializing-alembic&quot;&gt;Installing &amp;amp; initializing Alembic&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;installing alembic&lt;/strong&gt;
Alembic is installed like any other Python package, you install it using your package manager. using uv you might do:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; add alembic&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If you&#39;re not using uv (which will handle virtual environments automatically), make sure you create and activate your virtual environment . while you could install it system-wide, It is more common to install it in virtual environment as it will use  SQLAlchemy and database drivers that are more appropriate for local installations.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;initializing alembic&lt;/strong&gt;
Once installed, initialize Alembic with the init command. Run this in the directory where your application code lives.
The reason is that Alembic invokes env.py whenever you run commands, and that script needs to be able to import your SQLAlchemy models. If your Python path is not set correctly,
Alembic will not find them.
initialize alembic with&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;alembic&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; init alembic&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;After running  that command, Alembic will create a directory called alembic/ (you can change the name if you want, but most people leave it as the default).&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;-&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; versions/&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # where your revision files will go (it will be empty at first)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;-&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; env.py&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # the Python script that will run whenever you invoke alembic command&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;-&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; script.py.mako&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # a template used to generate new revision files(the default one will be what Alembic call generic template)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;-&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; alembic.ini&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # configuration file at the root of your project&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;h2 id=&quot;alembic-ini-env-py&quot;&gt;alembic.ini &amp;amp;&amp;amp; env.py&lt;/h2&gt;
&lt;p&gt;Amongst the files alembic will create upon its initialization, lets focus on two important ones &lt;code&gt;alembic.ini&lt;/code&gt; and &lt;code&gt;env.py&lt;/code&gt;, alembic.ini file will be placed at the root of your project. This is the configuration file Alembic reads before doing anything else.
In practice, you won’t spend much time here, but it is worth knowing what it controls because it can bite you with weird errors if you ignore it entirely.
The are couple things in this file you can modify to your preference but one of the main thing here is sqlalchemy.url, which is your database connection string. as you run your migrations env.py file will load the database url for connection from this file unless configured otherwiese.  but, you probably don&#39;t want to hardcode your database credentials in a file that gets committed, so in most cases you&#39;ll have to override this in env.py using environment variables.&lt;/p&gt;
&lt;p&gt;you can do this by loading your database url from your environment variables and using &lt;code&gt;set_main_option&lt;/code&gt; method on alembic config object.&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;config.set_main_option(&amp;quot;sqlalchemy.url&amp;quot;, db_url)&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;that will override the database url string to whatever you set your db_url you loaded from the environment variables. this is the one way you can do this.&lt;/p&gt;
&lt;h2 id=&quot;working-with-revisions-autogenerating-migrations&quot;&gt;Working with Revisions &amp;amp; Autogenerating Migrations&lt;/h2&gt;
&lt;p&gt;Once your environment is set up, you start creating migration scripts. You can write them by hand or let Alembic generate them for you.
When you run&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;alembic&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; revision -m &amp;quot;create users table&amp;quot;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;you get a new file in versions/ with empty upgrade() and downgrade() functions. At that point it&#39;s just a template that you will need to fill in what actually changes. This can be very useful when you want to add something new for example adding seed scripts or may be doing something complex that autogenerate wouldn&#39;t understand.
The other option is autogenerate. This is where Alembic looks at your SQLAlchemy models, compares them to what&#39;s actually in the database, and writes the migration scripts for you. But for this to work, Alembic needs to know two things: where your database is (that&#39;s the URL you configured in env.py), and what your models look like.
The models part requires you to set &lt;code&gt;target_metadata&lt;/code&gt; in &lt;code&gt;env.py&lt;/code&gt;. Assuming the application has declarative base you will import that and point it there. Though if you don&#39;t have the declarative base like in case you used sqlalchemy core API then you can import &lt;code&gt;metadata&lt;/code&gt; object. Don&#39;t forget to import all your models/tables you want the autogeneration to generate the script for(If you only import &lt;code&gt;Base/metadata&lt;/code&gt;, the autogeration will generate empty scripts):&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;from&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; app.models.mymodel import MyModel&lt;/span&gt;&lt;span&gt; &lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;from&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; myapp.models import Base&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;target_metadata&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; = Base.metadata&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now when you run&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;alembic&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; revision --autogenerate -m &amp;quot;add is_active column&amp;quot;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Alembic connects to your database, sees what tables and columns exist, compares that to your models, and generates a migration with the differences.
Here&#39;s the thing they tell us to keep in mind though, autogenerate is helpful but it&#39;s not magic. It&#39;s doing a diff, not understanding what you&#39;re trying to do. So, it misses renames , it&#39;ll see a column disappear and a new one appear, and think you want to drop one and add another. It also doesn&#39;t always catch constraint changes or custom types. And sometimes it generates weird stuff you didn&#39;t expect. For me I&#39;ve had it once try to drop PostGIS tables because they showed up in the database but not in my models, which would fail anyway because my user doesn&#39;t own those tables!
So, we should always look at what it generated before we run it.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Autogenerate is not intended to be perfect!.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;running-migrations&quot;&gt;Running migrations&lt;/h2&gt;
&lt;p&gt;When you&#39;re ready to apply migrations, you run&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;alembic&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; upgrade head&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;to apply everything pending, or&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;alembic&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; upgrade&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; &amp;lt;&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;revision_i&lt;/span&gt;&lt;span&gt;d&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To go to a specific revision. What actually happens here is pretty straightforward. Alembic connects to your database and checks for a table called &lt;code&gt;alembic_version&lt;/code&gt;. If it doesn&#39;t exist, it creates it. This table has one column and one row  that stores the current revision ID. That&#39;s almost the entire tracking mechanism!.
Alembic reads that value, loads all the revision files from versions/, and figures out which ones need to run to get from where you are to where you want to be. Then it runs the upgrade() function in each file, one by one, in order. After each one succeeds, it updates alembic_version with the new revision ID.
If something fails during a migration, Alembic stops and doesn&#39;t update the version table. So when you fix the problem and run it again, it picks up from where it left off!.
The &lt;code&gt;alembic_version table&lt;/code&gt; is simple but it&#39;s important. avoid editting it manually. If you need to mark a migration as applied without actually running it , maybe you applied it by hand or something , use&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89B4FA;font-style: italic;&quot;&gt;alembic&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt; stamp&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; &amp;lt;&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;revision_i&lt;/span&gt;&lt;span&gt;d&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;instead of touching the table directly.&lt;/p&gt;
&lt;h2 id=&quot;wrapping-up&quot;&gt;wrapping up&lt;/h2&gt;
&lt;p&gt;There are more to alembic but this workflow and commands are what I have realized I use more often, I hope you got something out of this.&lt;/p&gt;
&lt;p&gt;bye!&lt;/p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>Datetimes, Timezones, and UTC in Python3 (Naive vs Aware Explained)</title>
        <published>2025-11-23T00:00:00+00:00</published>
        <updated>2025-11-23T00:00:00+00:00</updated>
        
        <author>
          <name>NIYONSHUTI Emmanuel</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://emmanuelniyonshuti.github.io/blog/datetimes-timezones-and-utc-in-python/"/>
        <id>https://emmanuelniyonshuti.github.io/blog/datetimes-timezones-and-utc-in-python/</id>
        
        <content type="html" xml:base="https://emmanuelniyonshuti.github.io/blog/datetimes-timezones-and-utc-in-python/">&lt;h2 id=&quot;what-are-they&quot;&gt;What are they?&lt;/h2&gt;
&lt;p&gt;A date is the combination of a year, month and day. A time is an hour, minute, seconds and optionally microseconds. A datetime (or date and time) represents both combined together.&lt;/p&gt;
&lt;p&gt;When you have a datetime, it&#39;s representing a specific moment in time. Like for example, we are on 11/22/2025 at 1:25:45. That&#39;s a datetime and it&#39;s obvious to everyone. But we know all regions on earth are not on the same timezone, right? So that datetime represents that specific moment at where you are.&lt;/p&gt;
&lt;p&gt;Here&#39;s where it gets tricky: the numbers 1:25:45 by themselves don&#39;t tell you the actual moment in time. If I&#39;m in Kigali and you&#39;re in New York, and we both write down &quot;1:25:45&quot;, we&#39;re talking about different moments. Your 1:25 PM happens hours before mine. The numbers are the same, but the actual point in time is different.&lt;/p&gt;
&lt;h2 id=&quot;how-python-sees-datetimes&quot;&gt;How Python sees datetimes&lt;/h2&gt;
&lt;p&gt;In Python, they distinguish datetimes into two categories:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Naive&lt;/strong&gt; - A datetime that has no information about its timezone. Whatever timezone (e.g., UTC, local time) that datetime represents, it is up to the program running it to decide that.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Aware&lt;/strong&gt; - A datetime that knows what timezone it represents.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;The Python documentation puts it like this&lt;/em&gt;:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;a naive object does not contain enough information to unambiguously locate itself relative to other date/time objects. Whether a naive datetime represents UTC, local time, or time in some other timezone is purely up to the program.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span&gt; UTC&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Naive datetime - no timezone info&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;naive_dt&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;2025&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 11&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 22&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 13&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 25&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 45&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;naive_dt&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;tzinfo&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # None&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Aware datetime - has timezone information&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;aware_dt&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt;2025&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 11&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 22&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 13&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 25&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #FAB387;&quot;&gt; 45&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span style=&quot;color: #EBA0AC;font-style: italic;&quot;&gt; tzinfo&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt;=&lt;/span&gt;&lt;span&gt;UTC&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;aware_dt&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span&gt;tzinfo&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # datetime.timezone.utc&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You can see the &lt;code&gt;tzinfo&lt;/code&gt; attribute of a Python datetime object holds the timezone information. When it is &lt;code&gt;None&lt;/code&gt;, the datetime is naive. When it has a value, it&#39;s aware.&lt;/p&gt;
&lt;h2 id=&quot;why-does-this-matter&quot;&gt;Why does this matter?&lt;/h2&gt;
&lt;p&gt;The problem with naive datetimes is that Python doesn&#39;t know what they represent. If you store a naive datetime in a database, later when you read it back, you have no idea if those numbers mean UTC, local time, or something else.&lt;/p&gt;
&lt;p&gt;With aware datetimes, Python knows exactly what moment in time you&#39;re talking about. It can convert between timezones correctly because it knows where you started.&lt;/p&gt;
&lt;p&gt;Now, naive datetimes aren&#39;t inherently wrong. Say your application only runs in one timezone, your server is configured with that timezone, and you&#39;re not dealing with users across different locations , naive datetimes can work fine in that situation. But, the moment you need to work across timezones, you need aware datetimes.&lt;/p&gt;
&lt;h2 id=&quot;what-is-utc&quot;&gt;What is UTC?&lt;/h2&gt;
&lt;p&gt;I wanted to also talk a little bit about UTC (Coordinated Universal Time) in case it confuses you!. This is a universal reference point for time. It&#39;s not a timezone that belongs to any location, it&#39;s the zero point that all other timezones are measured against.&lt;/p&gt;
&lt;p&gt;For example, when you say &quot;Kigali is UTC+2&quot;, that means Kigali time is 2 hours ahead of UTC. When it&#39;s 13:00 in UTC, it&#39;s 15:00 in Kigali.&lt;/p&gt;
&lt;p&gt;In programming, the more common practice is to store time in UTC and convert to other timezones only when displaying to users. This way, you always know exactly what moment in time you&#39;re working with.&lt;/p&gt;
&lt;h2 id=&quot;the-common-workflow&quot;&gt;The common workflow&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;1. Getting the current time&lt;/strong&gt;&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span&gt; UTC&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Get current time in UTC (aware)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;utc_now&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;now&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;UTC&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;utc_now&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # 2025-11-22 11:25:45.123456+00:00&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You might have seen &lt;code&gt;datetime.utcnow()&lt;/code&gt; in older code. This has been deprecated because it returns a naive datetime even though it represents UTC time. Meaning it gives you UTC datetime but that datetime is naive , it doesn&#39;t have timezone information, which can be confusing.&lt;/p&gt;
&lt;p&gt;Also, be aware that using &lt;code&gt;datetime.now()&lt;/code&gt; without arguments returns naive datetime in your system&#39;s timezone:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# This returns naive datetime&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;naive_now&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;now&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;()&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;  # naive, system timezone&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;strong&gt;2. Converting between timezones&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;To convert between timezones, you use &lt;code&gt;astimezone()&lt;/code&gt;. This only works on aware datetimes:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span&gt; UTC&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; zoneinfo&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; ZoneInfo&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Start with UTC&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;utc_time&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;now&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;UTC&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Convert to Kigali timezone&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;kigali_tz&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt; ZoneInfo&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;color: #A6E3A1;&quot;&gt;&amp;quot;Africa/Kigali&amp;quot;&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;kigali_time&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span&gt; utc_time&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;astimezone&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;kigali_tz&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;utc_time&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;    # 2025-11-22 11:25:45+00:00&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #FAB387;font-style: italic;&quot;&gt;print&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;kigali_time&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt; # 2025-11-22 13:25:45+02:00&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The actual moment in time is the same, but the numbers changed because we&#39;re viewing it from a different timezone.&lt;/p&gt;
&lt;p&gt;This works when you know which timezone to convert to. But here&#39;s the thing — in most cases, you don&#39;t actually know the timezone for every user. You could ask them to set it in their profile, or try to infer it from their location, but that&#39;s not always practical.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;3. The practical approach: let the client handle it&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;What you usually do is keep your aware datetime in UTC, send it in your API response, and let the client (like a browser) handle the conversion to local time. Browsers know the user&#39;s timezone, so they can convert UTC to local time automatically:&lt;/p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #CDD6F4; background-color: #1E1E2E;&quot; &gt;&lt;code data-lang=&quot;python&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt;from&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #CBA6F7;&quot;&gt; import&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;,&lt;/span&gt;&lt;span&gt; UTC&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Your backend: store and send UTC&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;utc_time&lt;/span&gt;&lt;span style=&quot;color: #94E2D5;&quot;&gt; =&lt;/span&gt;&lt;span&gt; datetime&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color: #89B4FA;&quot;&gt;now&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;(&lt;/span&gt;&lt;span&gt;UTC&lt;/span&gt;&lt;span style=&quot;color: #9399B2;&quot;&gt;)&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Send this in API response: &amp;quot;2025-11-22T11:25:45+00:00&amp;quot;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Browser JavaScript handles conversion, example:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# new Date(&amp;quot;2025-11-22T11:25:45+00:00&amp;quot;).toLocaleString()&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9399B2;font-style: italic;&quot;&gt;# Shows in user&amp;#39;s local timezone automatically&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is simpler and more reliable than trying to manage every user&#39;s timezone on the server. Your server works with UTC, your client converts to local time.&lt;/p&gt;
&lt;h2 id=&quot;summary&quot;&gt;Summary&lt;/h2&gt;
&lt;p&gt;A datetime is just numbers representing a moment in time, but without timezone information, those numbers are ambiguous. Python&#39;s naive datetimes lack timezone information and can cause bugs when working across timezones. Aware datetimes explicitly know what timezone they represent, making them safe to store and convert. The common practice is to store in UTC and convert to user timezones only for display.&lt;/p&gt;
</content>
        
    </entry>
</feed>
