Clojure Bites - Mazeboard 5 - More on actions, tests, schema and future plans

Unknown — Sun, 31 Dec 2023 00:00:00 +0000
Intro</h1>
With the latest changes, described here</a>, we have come to a point were game and UI logic communicate with actions and event, using clojure.core.async</a> as an abstraction of the communication channel, the approach can be briefly described as:

UI send actions to the game logic</li>
Game logic handle the actions, emit the events and change its state</li>
UI logic handle events, changing its state and view accordingly</li> </ul>
For now, all of this is happening inside the same browser but given that we have already in place and abstraction of the communication channel, it should not be hard to put a network between the two layers.
Before moving to that topic, though, few more changes are needed. There is still quite some hardcoded stuff, for example the game state, the user actions, the players data just to name the more relevant ones, so first things to address will be:

calculate the valid `:move</code> actions by looking at the player position in the board</li>`
`simulate user actions instead of generating events based on hardcoded game state</li>`
start from a plain game state a build it with actions (starting with simulated ones, for now)</li> </ul> At this point the base infrastructure will be more or less ready and I think it will be a good time to start adding some tests because, as the game logic will grow, adding more actions and events, it will be easier to catch regressions or to refactor some bits of the project (very likely to happen). The same applies to the schema of actions and events, it would be nice to validate what goes back and forth in the communication channels.I'll close this update reasoning about a practical approach to create turn based, multiplayer games. If you are curious jump there</a> directly. Calculating valid moves</h1> Up until now, after a user action the next one was hardcoded, if I want to make a real game then the next possible actions must be calculated based on it is actually possible, for example move to east only if the door to east is open. This is based on where the player is positioned on the board. (defn valid-move-actions [target board position] (->> (get-in board [:tiles position]) tile/open-walls (mapv (fn [direction] [:move-player {:direction direction :target target}])))) </code></pre> In this case we calculate the valid :move</code> actions based on the open walls of the tile where the player is in. As we will add more actions we will have to do something similar. The nice thing is that now it is possible to use both players when developing the game locally! Simulate user actions</h1> Now that it is possible to generate the available actions based on what the player can actually do, it is time to move out the hardcoded events and only use player actions to update the game state, at this stage the actions that we want to (or are able to) support are: create game</li> join game</li> play action</li> </ul> To simplify, the board layout will still be hardcoded, generating a random board will be addressed in a later post. We will change from a predefined game state and events to a plain state and simulating user actions to create and start a game; it can seem like an unnecessary step but I'd argue that it can help to identify how the players will interact with the game, making the code closer to the final shape. Given that after initializing the game we already have the actions channel, simulating player actions is as simple as: (defn simulate-first-user-actions [actions-chan] (go (>! actions-chan [:create-game]) (>! actions-chan [:join-game {:name "one"}]) (<! (timeout 500)) (>! actions-chan [:join-game {:name "two"}]) (<! (timeout 500)) (>! actions-chan [:start-game]))) </code></pre> Tests</h1> At this point we have tested the new game layout manually, as we move forward, adding more actions and events it will be harder to keep track of all code paths and use case, so I think it is time to add some tests to help us move forward. People familiar with the TDD would argue that tests would have helped earlier because we can exercise the internal API sooner, discovering weaknesses (in its design or implementation) but at the same time I had no clear idea about how the code would have evolved; what is the value of a well tested bad code design? This is clearly personal taste, so use whatever work for you. I am keeping most of the code in cljc files, because most of it can be used in different runtimes, even if at this time I am more interested in the JS side of it. shadow-cljs has a basic test runner that is more than enough for my needs, I don't even need to address the browser at this point, at least for the game logic, and I will leverage this opportunity by having a nodejs</code> target test alias in the shadow-clj.edn</code> project file, that look like this: :builds {;;; ... other build targets :tests {:target :node-test :output-to "test-out/tests.js" :autorun true} } </code></pre> Now I can run npx shadow-cljs watch game tests</code> and have the tests to run each time I save sources and tests, which is quite convenient! Shadow-Cljs has a lot of configuration options, even for tests, so I'd suggest you to take a look at the docs</a> to get familiar with what it offers. It is worth mentioning that in this case I am targeting nodejs as the runtime and not the browser, this is because the code that I am testing is not related to the DOM but just the actions to events game loop; when or if it will come the time to test the UI I'll have to target the browser, which is something I still have to look at but it is well supported by shadow-cljs, again refer to the extensive documentation if you want to learn how it works. Now the project is ready for the next step, that is to make it playable by multiple players. Multiplayer on the server, or not?</h1> So far the game logic and UI are running in the same process, and communicating via core.async/chan</code>, the following diagram shows how it works. This is convenient while developing but there is no way to play with other people! A general approach for multiplayer games is to have a server somewhere which will handle the game logic and updates all clients when something happens. Given the current setup of the code this could be easily achieved by running the actions->events game loop in a web server which would receive actions via WebSocket and broadcast events using the same WebSocket connection; another approach could be to have a endpoint where players can send actions (and possibly receive validation errors) and broadcast events via SSE. My favorite setup would include: Aleph</a> web server to handle calls to the actions endpoint and broadcast events via SSE</li> Reitit</a> to define the endpoints and handle the content negotiation</li> Malli</a> to define and validate actions and events schema</li> Bonus, but not really needed at this point, some durable storage to store the state of the game; a relational DB plus HoneySQL</a> are battle tested solutions but lately I am having fun with datalog DBs. I will see which direction to take when it will come the time to work on this part.</li> </ul> Here is a diagram of how the this client server setup may look like: Another approach I am considering is inspired by how early multiplayer games worked back in the past century, in the nineties (oh my, how old am I...). In that setup one instance of the game works both as a client and a server, other players in the same network (or with a direct connection) connect their game client to the remote instance of the game which is responsible of updating the "official" game state and sending updates to the clients. One nice property of this setup is that clients could reduce the latency of the network by pre-computing the next state of the game using the same code which was running at the server side and possibly adjust the state later if it diverged from what the server said it should have been. This method is called client side prediction, I think it has been invented at ID Software. This worked quite well and did not require a dedicated server to play the game, just the players' PCs and some networking gear. At that time I had few (cheap) network cards that I could lend to friends plus the cables</a> to setup our LAN and play. Today, mostly everyone who can use a computing device can access the internet in a way or another so the network part can be considered as generally available. Can we replicate this approach with browser based games? After all the game logic can run in the browser and the UI layer can communicate with the game logic via a network abstraction, so it looks like that we can at least dream about not needing a full featured server to run our multiplayer game, all we need is a way to let all the instances of the game to talk! What do have we today (end of 2023) to implement this approach? First option is WebRTC</a> which gives us the ability to create peer to peer connections between the players. It is available for the browsers in the form of Javascript API and Android and iOS in the form of libraries; the only, minor, downside is that to connect two peers it requires a server side service for peer discovery and NAT traversing</a>. This is an interesting option that I will explore in future. The second option is to coordinate the communication between the players with a simple web app that can receive actions from a POST endpoint and broadcast events via SSE. The benefit is simplicity (at least if you are already familiar with this tech) and control which will come in handy later, for example to add authentication, persistent storage and so on. To be honest, how the players will communicate is just an implementation detail and it does not change how the architecture is designed: The full Excalidraw</a> canvas is available here</a>. Conclusions</h1> Now that the multiplayer architecture is more or less designed it is time to start thinking about how to implement the thin communication layer. With a network layer it will become more relevant to have a well described actions and events schema, for which I'll use malli</a>. The game logic must also be completed to have a full playable game but this can leverage the local development setup, without the complication of a network; fortunately both aspects can be developed in parallel and are not blocking each other. I am happy to close this year with all the foundation already laid out, Happy New Year everyone! Discuss</h1> Mastodon</a></li> Reddit</a></li> Clojurians</a></li> Twitter</a></li> Linkedin</a></li> </ul> Mazeboard related posts</h1> Mazeboard 0</a> - Project intro</li> Mazeboard 1</a> - Dumdom event handler</li> Mazeboard 2</a> - Using core async to decouple UI and game logic</li> Mazeboard 3</a> - Improving UI and game logic decoupling</li> Mazeboard 4</a> - Replacing commands with events</li> Mazeboard 5</a> - More on actions, adding tests and future multiplayer architecture</li> </ul>



Clojure Bites - Mazeboard 4 - From commands to events for a better separation of concerns
Unknown — Tue, 19 Dec 2023 00:00:00 +0000
Intro</h1>
Last post</a>
was about the separation of game and UI logic, where game and UI state have been
made independent by turning user actions to commands that would have updated
both game and UI state. That was possible because both state maps share the same
structure; even if convenient initially (you use the same code for both) it can 
be limiting in the long run, who knows how the next UI will be rendered tomorrow?
It is also a matter of separation of concerns, the current schema has data that is
not needed for the game logic, for example some element classes, and the same
applies to the UI layer.</p>
From commands to events</h1>
Given that the final goal is to have independent state schema in the game and UI
logic the approach based on shared commands which will update both states would
not work anymore; the new approach instead will be based on events which will be
responsible for describing what happened and then each layer will handle them in a
way that best fits their needs (for example updating the view, triggering animations
etc in the UI layer). We will transition from a game loop like the following:</p>

Game wait for actions</li>
UI send the action after a user click</li>
Game translates actions to commands</li>
Game apply commands to its state</li>
Game send commands to listeners (UIs)</li>
UIs apply commands to their state</li>
</ul>
To something like:</p>

Game wait for actions</li>
UI send the action after a user click</li>
Game translate actions to events, calculating the next state (for example new player position, new round number etc)</li>
Game interpret the events to update its state</li>
Game send events to listeners</li>
Listeners interpret the events to update their state</li>
</ul>
The event schema can be quite simple, for example a tuple with the event keyword
and an optional map holding the information attached to it:</p>

</span>(</span>def </span>player-moved-event </span>[</span>:player-moved </span>{</span>:target 0 :old-position </span>{</span>:row 0 :col 0</span>} </span>:new-position </span>{</span>:row 0 :col 1</span>}}])
</span>
</span></code></pre>
Easy to build, easy to destructure.</p>
One example</h1>
Let's have a look at a concrete example, currently the only available action
is :move-player</code> so yeah we don't much choice :)</p>
The action itself is simple:</p>

</span>[</span>:move-player </span>{</span>:direction :east :target 0</span>}]
</span>
</span></code></pre>
It means that the user 0 can move to east, amazing! If the game logic will
receive this action it will generate two events, :player-moved</code> to signal
the new position of the user and round-started</code> to signal that a new round
has started. For now the function that translates actions to events looks like
this:</p>

</span>(</span>defn </span>action->events
</span>  [game action]
</span>  (</span>prn </span>"</span>Triggered action</span>" action)
</span>  (</span>match</span> action
</span>         [</span>:move-player</span> opts] [(</span>player-moved-event</span> game opts)
</span>                              (</span>round-started-event</span> game)]))
</span>
</span></code></pre>
Each event will describe what happened, the game logic will do its best to
reflect the change to its state and the UI will interpret the event to show
the change to the user; it is more of a mindset that is applied to both layers
and this helps to keep both worlds separated. Maybe it is worth having a look at
how these first two event generators are implemented:</p>

</span>(</span>defn </span>player-moved-event
</span>  [{</span>:keys </span>[players]}
</span>   {</span>:keys </span>[direction target]}]
</span>  (</span>let </span>[old-pos (</span>get-in</span> players [target </span>:position</span>])
</span>        new-pos (</span>update-position</span> old-pos direction)]
</span>    [</span>:player-moved </span>{</span>:target</span> target
</span>                    </span>:old-position</span> old-pos
</span>                    </span>:new-position</span> new-pos}]))
</span>
</span>(</span>defn </span>round-started-event
</span>  [{</span>:keys </span>[turn players]}]
</span>  (</span>let </span>[{</span>:keys </span>[current-player round-number]} turn]
</span>    [</span>:round-started </span>{</span>:current-player </span>(</span>mod </span>(</span>inc</span> current-player) (</span>count</span> players))
</span>                     </span>:round-number </span>(</span>inc</span> round-number)}]))
</span>
</span></code></pre>
Now that we have events these must be interpreted to reflect these changes
to the respective states, let's start with the game side:</p>

</span>(</span>defn </span>handle-event-player-moved
</span>  [game {</span>:keys </span>[target new-position]}]
</span>  (</span>assoc-in</span> game [</span>:players</span> target </span>:position</span>] new-position))
</span>
</span>(</span>defn </span>handle-round-started
</span>  [game turn]
</span>  (</span>assoc</span> game </span>:turn</span> turn))
</span>
</span>(</span>defn </span>handle-event
</span>  [game event]
</span>  (</span>prn </span>"</span>Received event </span>" event)
</span>  (</span>match</span> event
</span>         [</span>:player-moved</span> opts] (</span>handle-event-player-moved</span> game opts)
</span>         [</span>:round-started</span> opts] (</span>handle-round-started</span> game opts)))
</span>
</span></code></pre>
handle-event</code> will dispatch to the proper function which then will
update the game state. One thing I like about this approach is that
it makes state transitions quite straightforward at every layer.</p>
p.s. Yes yes, the name of the functions is not consistent at all, I will take
care of this in future.</p>
How does it look on the frontend side? Different! Which is the point of
this change, here is how player-moved</code> is handled in the UI:</p>

</span>(</span>defn </span>event--player-moved
</span>  [game {</span>:keys </span>[target old-position new-position]}]
</span>  (</span>-></span> game
</span>      (</span>update-in </span>[</span>:board :tiles</span> old-position </span>:players</span>] disj target)
</span>      (</span>update-in </span>[</span>:board :tiles</span> new-position </span>:players</span>] conj target)
</span>      ))
</span>
</span></code></pre>
So what is the difference? At the game logic layer we just need to update
the position in the player map and at the UI layer we "put" the player id
in the new tile and we remove it from the previous position.</p>
Conclusions</h1>
In this short post we have explored how to make the game and UI states really
independent of each other. Still, the initial map is the same but then it will
evolve following two distinct paths. It is not just a cosmetic change but it 
creates the baseline for different client implementations, for example I am 
curious to try PixiJS (again) or even Three.js and this setup would make it
possible to just work on the UI layer without touching the game logic.</p>
Discuss</h1>

Reddit</a></li>
Linkedin</a></li>
Twitter</a></li>
</ul>
Code</h1>
This post include changes made in two different merge requests:</p>

first switch to events (MR</a>)</li>
build the UI state gradually using events (MR</a>)</li>
</ul>
Mazeboard related posts</h1>

Mazeboard 0</a> - Project intro</li>
Mazeboard 1</a> - Dumdom event handler</li>
Mazeboard 2</a> - Using core async to decouple UI and game logic</li>
Mazeboard 3</a> - Improving UI and game logic decoupling</li>
Mazeboard 4</a> - Replacing commands with events</li>
Mazeboard 5</a> - More on actions, adding tests and future multiplayer architecture</li>
</ul>


Clojure Bites - Mazeboard 3 - core.async to update the UI layer
Unknown — Sat, 09 Dec 2023 00:00:00 +0000
Intro</h1>
In the last post</a>
we have started decoupling game and UI logic in order to be able
to run the two layers in different processes; we are not there
yet and it will take some time to get to the final client/server
setup, but we will get there, eventually :)</p>
We have moved some logic and the shared state to the game layer,
even if the UI is still using the state atom from the mazeboard.game</code>
namespace, actions are sent to game layer using core.async</code> instead
of calling the update functions directly; it is not much but it is
honest work.</p>
Goal of this post is to further decouple UI and game logic by having
a separate state for both layers and propagate changes from the game
layer to the UI via core.async</code>.</p>
Recap of the previous changes</h1>
We started from something like this:</p>

UI is rendered using dumdom components driven by a map describing the game state</li>
DOM events like clicks are handled by dumdom's global event handler, again data driven</li>
data associated with events describe actions that the player wants to do like moving in the board</li>
actions are handled by directly calling the game layer which updates the global state</li>
the UI is subscribed to the global state and re-renders when the state atom changes</li>
</ul>
And we ended up with the new setup:</p>

the shared have been moved from maeboard.ui.core</code> to mazeboard.game</code> namespace</li>
actions are sent to the game layer using a core.async</code> channel which the game layer listens to, to break the direct call dependency</li>
</ul>
The UI still listen to the game state atom directly, and we are going to fix this.</p>
The commit is available here</a>.</p>
Two layers, two state atoms</h1>
We want to use two distinct state atoms for:</p>

managing and mutating the game state according to requested actions</li>
update the UI state that drives the rendering of the game board</li>
</ul>
The most obvious thing to do would be to keep the pre existing game
state atom in the mazeboard.game</code> namespace and a dedicated state 
atom to mazeboard.ui.core</code> namespace so that the two layers can 
happily live their independent lives :) </p>
This is not enough, because we want to be able to update the UI state
after something happened in the game layer; this must somehow be communicated
to the UI layer via core.async</code> channels but given that now we are directly
"assoc"ing or "update(-in)"ing directly to the game state map, it is not easy
to transfer the intended changes between layers.</p>
Let's do a step back, or sideways, to try do define a data format that 
can describe state changes that can be transferred between layers and
applied accordingly to both states.</p>
Abstracting state changes</h1>
We can start by having a look at how the action that handles player's 
movement is implemented:</p>

</span>(</span>defn </span>action-move-player
</span>  [{</span>:keys </span>[current-player players] </span>:as</span> game} {</span>:keys </span>[direction target]}]
</span>  (</span>let </span>[old-pos (</span>get-in</span> players [target </span>:position</span>])
</span>        new-pos (</span>update-position </span>(</span>get-in</span> game [</span>:players</span> target </span>:position</span>]) direction)]
</span>    (</span>-></span> game
</span>        (</span>assoc-in </span>[</span>:players</span> target </span>:position</span>] new-pos)
</span>        (</span>assoc </span>:current-player </span>(</span>mod </span>(</span>inc</span> current-player) (</span>count</span> players)))
</span>        (</span>update-in </span>[</span>:board :tiles</span> old-pos </span>:players</span>] disj target)
</span>        (</span>update-in </span>[</span>:board :tiles</span> new-pos </span>:players</span>] conj target)
</span>        (</span>update </span>:round-number</span> inc))))
</span>
</span></code></pre>
All it is doing is to take the game map, and applying assoc(-in)</code> and 
update(-in)</code> functions to it; update</code> and update-in</code> can be replaced by
their assoc forms if handled correctly so, if we can replace the direct calls
to assoc(-in) functions to data then we then can abstract away what we want to do
using just data in a form the looks something like:</p>

</span>[[</span>:assoc-in </span>[</span>:players</span> target </span>:position</span>] new-pos]
</span> [</span>:assoc-in </span>[</span>:board :tiles</span> old-pos </span>:players</span>] old-tile-players]
</span> [</span>:assoc-in </span>[</span>:board :tiles</span> new-pos </span>:players</span>] new-tile-players]
</span> [</span>:assoc :current-player </span>(</span>mod </span>(</span>inc</span> current-player) (</span>count</span> players))]
</span> [</span>:assoc :round-number </span>(</span>inc</span> round-number)]]
</span>
</span></code></pre>
We can then have a function that takes a data structure, the requested
changes and that can spit out a new, updated data structure:</p>

</span>(</span>defn </span>apply-commands
</span>  [state commands]
</span>  (</span>reduce
</span>   (</span>fn </span>[game command]
</span>     (</span>match</span> command
</span>            [</span>:assoc</span> key value] (</span>assoc</span> game key value)
</span>            [</span>:assoc-in</span> path value] (</span>assoc-in</span> game path value)))
</span>    state commands))
</span>
</span></code></pre>
With all these building blocks in place what we have left to do is to
update our code to:</p>

transform actions (like playing a move) to commands to update the game state</li>
broadcast all these commands to interested parties (game logic and UI)</li>
apply the commands to reflect the changes</li>
</ul>
The final result can be seen here</a></p>
Changes can be summarized as following:</p>

Game and UI layers have their own state</li>
On start, the initial game state is used in both layers</li>
On user actions a new changeset, in form of commands, is generated</li>
The commands are applied to the game state</li>
The commands are sent to the UI layer and applied there too</li>
</ul>
Conclusions</h1>
We have started with a tightly coupled core and UI logic code base,
slowly we have pushed to the edges the concerns that are more suited 
to each layer; communication between layers is based on core.async</code> to
help us to reason about intents (user actions or state changes) and we
have closed the circle with a setup that leverages the abstractions that
we have built to give us the possibility to reason about game and UI independently.</p>
It is not 100% evident at this point, but the UI and game state are still sharing 
way too much structure:</p>

there is no reason why the game state must be concerned with actions available in a tile</a></li>
there is no reason why the UI state must be concerned by the :players</code> key in the game map, given that it does not make any use of it</li>
</ul>
This will be addressed in the next post, but for now I am happy with what we
have achieved!</p>
Mazeboard related posts</h1>

Mazeboard 0</a> - Project intro</li>
Mazeboard 1</a> - Dumdom event handler</li>
Mazeboard 2</a> - Using core async to decouple UI and game logic</li>
Mazeboard 3</a> - Improving UI and game logic decoupling</li>
Mazeboard 4</a> - Replacing commands with events</li>
Mazeboard 5</a> - More on actions, adding tests and future multiplayer architecture</li>
</ul>


Clojure Bites - Mazeboard 2 - Using core.async to decouple game and UI logic
Unknown — Tue, 05 Dec 2023 00:00:00 +0000
Intro</h1>
The previous post</a> ended
by looking at the shortcomings of having the game and UI logic completely coupled,
which is not ideal in a client/server game setup. We want to setup our code to make it 
easy to reason about the two layers separately, focusing on their specificity and,
of course, we want to prepare it make it possible for the two layers to communicate
over a network connection.</p>
Here is the plan:</p>

try not to break the ability to show the board and trigger one action (you know, for confidence)</li>
move the app state to another namespace to highlight dependencies</li>
break the direct call graph by introducing a sorts of communication layer</li>
let the game to logic handles actions</li>
UI logic applies changes directed by the core game logic</li>
</ul>
Move slowly without breaking everything</h1>
Baby steps. I am not Meta and I cannot move fast and break everything ;)</p>
One thing to keep in mind is that everything is still running inside the
same process in the browser, but we want to slowly separate client and
server concerns so that when the server will run in a remote process then
the client will not need any fundamental changes to keep it working.</p>
The current setup has a global state that represents the game (players,
board with the tiles, current turn and possible actions) and it is used
to render the UI components and handle the game logic altogether; before
splitting the two concerns lets move this state elsewhere so that we can
start to identify the dependencies between the two layers. Here is
the new namespace that will hold the game logic, mazeboard.game</code>: </p>
(</span>ns</span> mazeboard.game
</span>  (</span>:require </span>[cljs.core.async </span>:as</span> async]
</span>            [mazeboard.actions </span>:as</span> actions]
</span>            [mazeboard.test-data </span>:as</span> test-data])
</span>  (</span>:require-macros </span>[cljs.core.async.macros </span>:refer </span>[go-loop]]))
</span>
</span>(</span>defonce </span>state </span>(</span>atom </span>{}))
</span>
</span>(</span>defn </span>game-loop
</span>  [actions-chan]
</span>  (</span>go-loop </span>[]
</span>    (</span>let </span>[action (</span>async/<!</span> actions-chan)]
</span>      (</span>swap!</span> state update </span>:game</span> actions/dispatch action))
</span>    (</span>recur</span>)))
</span>
</span>(</span>defn </span>init </span>[]
</span>  (</span>let </span>[actions-chan (</span>async/chan</span>)
</span>        events-chan (</span>async/chan</span>)]
</span>
</span>    (</span>swap!</span> state assoc </span>:game </span>(</span>test-data/create-game</span>))
</span>
</span>    (</span>game-loop</span> actions-chan)
</span>
</span>    [events-chan actions-chan]))
</span></code></pre>
The namespace contains a little bit more that just moving the state from
the UI layer to the logic layer, let's break it down:</p>

state</code> atom: holds the game state and the UI, for now, will subscribe to its changes</li>
init</code> fn: fills the state</code> atom with game data and returns two core.async</code> channels to communicate between the two layers</li>
game-loop</code> fn: will listen for actions sent to the actions-chan</code> channel and will update the game state accordingly, using the mazeboard.actions/dispatch</code> fn introduced in the previous post</li>
the events-chan</code> channel is not used at the moment but, in future, listeners of this channel will be able to update their state after changes of the game state</li>
</ul>
The namespace mazeboard.ui.core</code> requires few changes:</p>

remove (at least for now) its own state atom and watch the game state atom to render changes</li>
update the global dumdom event handler (for user actions) to send actions to the actions-chan</code> channel instead of calling the dispatch</code> action directly</li>
</ul>
Here is how the event handler setup look like now:</p>
  (</span>let </span>[[events-chan actions-chan] (</span>game/init</span>)]
</span>    (</span>d/set-event-handler!
</span>     (</span>fn </span>[_e actions]
</span>       (</span>doseq </span>[action actions]
</span>         (</span>put!</span> actions-chan action)))))
</span></code></pre>
Instead of subscribing to the mazeboard.game/state</code> atom directly it would have
been cleaner to return it from maeboard.game/init</code> fn and subscribe to that but
this setup is going to be changed soon so we can close an eye this time ;)</p>
After these changes the game renders and actions are handled correctly, baby steps!
Unfortunately there is still this one shared state that we want to get rid of, but
this will be covered in the next post.</p>
The full change set for this post is available here</a>.</p>
Conclusions</h1>
In this short post we have started the decoupling process of UI and game logic layers,
even if not completely, by using core.async</a>
as an abstraction of a communication layer.
The UI is still subscribed to a shared state (atom) to render updates but
at least the user actions are now sent "somewhere" instead of a direct function
call, preparing it for the next step of receiving game state updates using a 
similar approach.</p>
Discuss</h1>

Reddit</a></li>
</ul>
Mazeboard related posts</h1>

Mazeboard 0</a> - Project intro</li>
Mazeboard 1</a> - Dumdom event handler</li>
Mazeboard 2</a> - Using core async to decouple UI and game logic</li>
Mazeboard 3</a> - Improving UI and game logic decoupling</li>
Mazeboard 4</a> - Replacing commands with events</li>
Mazeboard 5</a> - More on actions, adding tests and future multiplayer architecture</li>
</ul>


Clojure Bites - Mazeboard 1 - Dumdom event handler
Unknown — Thu, 30 Nov 2023 00:00:00 +0000
Intro</h1>
In the previous</a> post I've introduced the
game I am working on, presenting the stack and the general code structure
I'd like to follow, I've closed the post with a couple of screenshot of a static
game board, now it is time to make it a bit more interactive.</p>
The goal is to let the user to click on one of the arrows (which represent the
available actions, more actions to come later) and move the player icon to correct
new position.</p>
The game UI state</h1>
The UI state is a map that holds the data that drives the rendering of the components,
and includes information about:</p>

the boards with its own tiles, tiles can also include which action can be performed on them</li>
the players with name and position</li>
the current player</li>
the current round</li>
the end position</li>
</ul>
The map will be updated after player actions and changes will be reflected in the interface
by re-rendering what have changed. To achieve this goal the UI state will be stored in an atom,
attaching the rendering function to the atom watch list, something like:</p>
(</span>ns</span> mazeboard.ui.core
</span>  (</span>:require </span>[dumdom.core </span>:as</span> d]
</span>            [mazeboard.test-data </span>:as</span> test-data]
</span>            [mazeboard.ui.components.app </span>:as</span> app]))
</span>
</span>(</span>defonce </span>state_ </span>(</span>atom </span>{}))
</span>
</span>(</span>defn </span>render </span>[state]
</span>  (</span>d/render </span>(</span>app/App</span> state)
</span>            (</span>js/document.getElementById </span>"</span>app</span>")))
</span>
</span>(</span>defn </span>init </span>[]
</span>  (</span>add-watch</span> state_ </span>:app </span>(</span>fn </span>[_ _ _ new-state] (</span>render</span> new-state)))
</span>
</span>  (</span>swap!</span> state_ assoc </span>:game </span>(</span>test-data/create-game</span>)))
</span></code></pre>
create-game</code> returns a map which is an implementation detail and can be ignored.
With this setup, each time the state_ atom will be mutated, the rendering function will
be called with the new state, causing a change in the UI. At this time the only way to 
make changes to the UI state is to manually swap!</code> or reset!</code> the atom, so not ideal
for a real game, but it is enough get started; this approach can be useful to manually
test ideas in the REPL until we are sure how something will be implemented, creating a
nice interactive development environment.</p>
Triggering actions on click events</h1>
After few iterations, playing with the game UI (man, I suck at CSS), I can shows where
the players are in the board and what the player "One" can do i.e. move south or east,
here is how it looks currently:</p>
</p>
Clicking on one of the arrows should trigger an action that will update the player's
position, change the current player, update the round number and update the UI to
reflect what has changed. First step is to handle clicks on the arrows.</p>
Dumdom provides two ways to attach event handlers</a>
to DOM elements, specifying the key :on-click</code> in the element configuration map:</p>

if the value of the key is a function then it will be called directly</li>
if the value is anything else (for example lists, vects, maps) then a global handler will be called</li>
</ul>
The global event handler must be registered with dumdom.core/set-event-handler!</code> whose only parameter
is a function that will receive the target element and whatever data has been specified in the
:on-click</code> value of the element. The event handler function have a signature like (fn [target data])</code>.</p>
Both methods (fn or data) have use cases that can be more or less suitable
depending on your specific needs, in this case I have opted for the data
driven approach for two reasons:</p>

it feels more declarative, the element "tells" what it wants to do</li>
it makes it possible to attach actions to elements from the game logic that can be later used by the UI; especially useful when the game logic will be handled by a separate (remote) process</li>
</ul>
Lets re-write the init method to setup an event handler:</p>
(</span>defn </span>init </span>[]
</span>  (</span>add-watch</span> state_ </span>:app </span>(</span>fn </span>[_ _ _ new-state] (</span>render</span> new-state)))
</span>  
</span>  (</span>d/set-event-handler!
</span>   (</span>fn </span>[_e actions]
</span>     (</span>doseq </span>[action actions]
</span>       (</span>prn</span> action))))
</span>
</span>  (</span>swap!</span> state_ assoc </span>:game </span>(</span>test-data/create-game</span>)))
</span></code></pre>
And here is how the Tile</code> component looks like, with data driven :on-click</code> trigger:</p>
(</span>defcomponent </span>Tile </span>[{</span>:keys </span>[actions position end-position? players]} all-players]
</span>  (</span>cond-> </span>[</span>:div.tile </span>{</span>:class </span>"</span>tile-background</span>"
</span>                      </span>:key</span> position}
</span>
</span>           (</span>for </span>[{</span>:keys </span>[on-click action-class]} actions]
</span>             [</span>:div.action </span>{</span>:on-click</span> on-click </span>:class</span> action-class}])
</span>
</span>           (</span>for </span>[player players]
</span>             (</span>let </span>[{</span>:keys </span>[class name]} (</span>nth</span> all-players player)]
</span>               [</span>:div.player </span>{</span>:class</span> class} name]))]
</span>
</span>    end-position?
</span>    (</span>conj </span>[</span>:div.end-position</span>])))
</span>
</span>;; example of component's actions
</span>#_[{</span>:on-click </span>[[</span>:move-player </span>{</span>:direction :east :target 0</span>}]]
</span>                </span>:action-class </span>"</span>action-move-east</span>"}
</span>   {</span>:on-click </span>[[</span>:move-player </span>{</span>:direction :south :target 0</span>}]]
</span>                </span>:action-class </span>"</span>action-move-south</span>"}]
</span></code></pre>
Dumdom does not impose any structure to the data that can be associated to
an element's event handler so I have taken the examples in Parens of the dead
and I will structure the actions as a vector of vectors, with the first element
of the sub vectors describing the action as a keyword and the second element as
the options/parameters map of the action itself.</p>
After setting up the global event handler, if we click on one the arrows, for
example to go east, in the browser console log we should see [:move-player {:direction :east :target 0}]</code>,
meaning that the event handler is working properly. But still, nothing is happening
in the game UI, lets fix that.</p>
Dispatching actions and state updates</h1>
We are now able to trigger actions and we need something that can update the game
state and reflect the changes to screen. In this case I think it is better jump
straight to the code that will move a player in Mazeboard:</p>
(</span>ns</span> mazeboard.actions
</span>  (</span>:require </span>[clojure.core.match </span>:refer </span>[match]]))
</span>
</span>(</span>defn </span>update-position
</span>  [position direction]
</span>  (</span>case</span> direction
</span>    </span>:north </span>(</span>update</span> position </span>:row</span> dec)
</span>    </span>:east </span>(</span>update</span> position </span>:col</span> inc)
</span>    </span>:south </span>(</span>update</span> position </span>:row</span> inc)
</span>    </span>:west </span>(</span>update</span> position </span>:col</span> dec)))
</span>
</span>(</span>defn </span>action-move-player
</span>  [{</span>:keys </span>[current-player players] </span>:as</span> game} {</span>:keys </span>[direction target]}]
</span>  (</span>let </span>[old-pos (</span>get-in</span> players [target </span>:position</span>])
</span>        new-pos (</span>update-position </span>(</span>get-in</span> game [</span>:players</span> target </span>:position</span>]) direction)]
</span>    (</span>-></span> game
</span>        (</span>assoc-in </span>[</span>:players</span> target </span>:position</span>] new-pos)
</span>        (</span>assoc </span>:current-player </span>(</span>mod </span>(</span>inc</span> current-player) (</span>count</span> players)))
</span>        (</span>update-in </span>[</span>:board :tiles</span> old-pos </span>:players</span>] disj target)
</span>        (</span>update-in </span>[</span>:board :tiles</span> new-pos </span>:players</span>] conj target)
</span>        (</span>update </span>:round-number</span> inc))))
</span>
</span>(</span>defn </span>dispatch
</span>  [game action]
</span>  (</span>prn </span>"</span>Triggered action</span>" action)
</span>  (</span>match</span> action
</span>         [</span>:move-player</span> opts] (</span>action-move-player</span> game opts)))
</span>
</span></code></pre>
The code must be read bottom up, first thing to look at is the dispatch</code> function
that will match the requested action to a specific function calling it; the fact the
here clojure.core.match</code> is being used is an implementation detail, you may prefer
a different approach (for example defmulti</code> is another good candidate). This is quite
simple, we try to match the requested action to something that we know how to handle,
in this case moving the player. action-move-player</code> will, of course, update the game
(and UI) state, returning it.</p>
There is still one bit missing, who is calling dispatch</code> and how this will update
the global state atom? We have to update the global handler to connect actions to
state updates, here is how the init fn is looking now:</p>
(</span>defn </span>init </span>[]
</span>  (</span>d/set-event-handler!
</span>   (</span>fn </span>[_e actions]
</span>     (</span>doseq </span>[action actions]
</span>       (</span>swap!</span> state_ update </span>:game</span> actions/dispatch action))))
</span>
</span>  (</span>add-watch</span> state_ </span>:app </span>(</span>fn </span>[_ _ _ state] (</span>render</span> state)))
</span>
</span>  (</span>swap!</span> state_ assoc </span>:game </span>(</span>test-data/create-game</span>)))
</span></code></pre>
After these changes, when clicking one of the arrows the player icon will
be placed correctly in the next tile, in this screenshot we can see the effect
of moving south</p>
</p>
Together with the updated player "One" position we can also observe that the round
number has changed and that now it is the turn of player "Two"; updating the tiles
to reflect what the other user can do will be the content of another post.</p>
Future improvements</h1>
At this point we have a working application that can react to user actions and render
the updated state accordingly. What I don't like about the current approach is that it
is coupling the game logic and the UI logic, for example knowing which actions are available
and how to render them is a concern of the UI state and not of the game state, but now
everything is mixed together and soon managing both the game and the UI will be a
complete mess and hard to maintain.
To make a concrete example here is how the tile is represented at the moment:</p>
(</span>def </span>tile </span>{</span>:actions </span>[{</span>:on-click </span>[[</span>:move-player </span>{</span>:direction :east :target 0</span>}]]
</span>                      </span>:action-class </span>"</span>action-move-east</span>"}
</span>                     {</span>:on-click </span>[[</span>:move-player </span>{</span>:direction :south :target 0</span>}]]
</span>                      </span>:action-class </span>"</span>action-move-south</span>"}]
</span>           </span>:walls </span>[</span>:closed :open :open :closed</span>]
</span>           </span>:end-position? true
</span>           </span>:players </span>[</span>0</span>]})
</span></code></pre>
The :actions</code>, :end-position?</code> and :players</code> keys are closely related to the UI, instead
the :walls</code> key is more on the game logic side.</p>
By separating game and UI I hope to get to a setup were:</p>

the UI will trigger actions that will be sent to the game layer</li>
the game layer will validate the requested action and, if valid, will update the game state</li>
after updating the game state, events will be triggered for all clients describing what has happened</li>
all clients will update their state using the received events and re-render the UI accordingly</li>
</ul>
A bonus benefit of this kind of separation of concerns is that unit testing the game logic and
the event generation that will feed the UI will cover most of the application!
(Thanks Parens of the dead</a> for this idea).</p>
Conclusion</h1>
In this post we have seen how to model the UI state (even if coupled with game state),
how to dispatch actions from DOM elements and how to handle them to update the game.</p>
The whole setup is quite primitive but good enough for a starting point. I am very
excited to finally decouple game and UI concerns, it will make it easier to build
game and UI logic separately.</p>
Discuss</h1>

Reddit</a></li>
Twitter</a></li>
Clojurians Slack</a></li>
</ul>
Sources</h1>
Mazeboard source code is available here</a>.</p>
Mazeboard related posts</h1>

Mazeboard 0</a> - Project intro</li>
Mazeboard 1</a> - Dumdom event handler</li>
Mazeboard 2</a> - Using core async to decouple UI and game logic</li>
Mazeboard 3</a> - Improving UI and game logic decoupling</li>
Mazeboard 4</a> - Replacing commands with events</li>
Mazeboard 5</a> - More on actions, adding tests and future multiplayer architecture</li>
</ul>


Clojure Bites - Mazeboard 0
Unknown — Sat, 18 Nov 2023 00:00:00 +0000
Intro</h1>
Welcome back to Clojure Bites!</p>
This is a new format compared to the previous issues, instead of focusing
on one specific topic, it will be more like a diary of what I am learning while
resurrecting an old game I was working on some time ago. After spending
a significant amount of time on the backend, I was stuck at the user facing part,
basically for two reasons:</p>

Lack of experience in frontend development</li>
Not knowing how to use WebSockets or SSE properly, especially how to structure the backend</li>
</ul>
The first problem was not a big one, with the help of Rum</a>
as the React wrapper and Citrus</a> for state management
I was making progress even if slowly. The realtime communication part instead was a big
blocker, it is ok if the frontend sucks as long as you can play the game! Time passed, I've
lost interest and the project has been parked until now; at least I've gained a bit of
experience with client/server realtime communication, lets see how far I will get this time :)</p>
The plan now is to, mostly, start from from scratch and implement the game
focusing on the frontend first, and back again to the backed later.
A great source of inspiration is Parens of the dead</a>,
I like how they have structured their code to have a thin (dump?) client backed by a fully
tested logic implemented in the backend; if something will look wrong or dumb in my
implementation I'll be the guy to blame, not them ;) </p>
The game</h1>
The game should be a sorts of rogue like board game where players explore a maze,
with the goal to find a treasure and bring it back to to starting position.
The board is composed by squares in a NxN grid, each square have randomly generated
connections to nearby squares which can be used to move towards the treasure and back.
Each turn players will flip a coin (or roll a dice) to get the possible action,
move to a next square or rotate a square of the board (useful to make it harder to 
reach the goal to other players).</p>
Frontend stack</h1>
The first change from the previous attempt is the build system, switching from
figwheel-main</a> to shadow-cljs</a>.
The main reason is that, to my understanding, it is easier to work with JS libraries
with shadow that with figwheel; in another project I was using PixiJS and the version
available in cljsjs repo was a bit outdated and I don't want to be in this situation again.</p>
Having covered the build system it is now time to pick something to render the game.
The first version of this project was built on Rum</a>
because it felt simple enough, Reagent</a>
was another (more popular) option that for some reason did not click for me, I totally
overlooked Quiescent</a> or Om(next)</a>.
Newer candidates in this space are Helix</a> or
UIx</a> if you after mature and up to date React wrappers,
dumdom</a> if you want to stick to virtual dom but
you are not interested/experienced in React, and finally htmx</a> if you
want to stick to the hypermedia concept. All are strong and valid options for one reason
or another but again I want to favor simplicity and dumdom being mostly focused on the
rendering part, looks like the best candidate for my needs.</p>
Last bit of tooling I am introducing to my stack is Portfolio</a>,
which is a sorts of Storybook</a> for ClojureScript, in few
words it makes it possible to test UI components in isolation and can be used as a documentation.</p>
Setting up the development environment</h1>
To recap, I am using shadow-cljs as the build system, dumdom as the rendering library and
portfolio to preview/debug components; the first step is to setup the project i.e.
create a shadow-cljs.edn:</p>
{</span>:dependencies </span>[[cjohansen/dumdom "</span>2023.10.13</span>"]
</span>                [no.cjohansen/portfolio "</span>2023.04.05</span>"]
</span>                [cider/cider-nrepl "</span>0.41.0</span>"]
</span>                [refactor-nrepl/refactor-nrepl "</span>3.9.0</span>"]]
</span>
</span> </span>:source-paths </span>["</span>src/cljs</span>" "</span>src/cljc</span>"] </span>;; where to look for sources
</span>
</span> </span>:dev-http </span>{</span>8080 </span>["</span>public</span>" "</span>classpath:public</span>"]} </span>;; "classpath:public" is required to server Portfolio's resources
</span>
</span> </span>:nrepl </span>{</span>:middleware </span>[cider.nrepl/cider-middleware
</span>                      refactor-nrepl.middleware/wrap-refactor]
</span>         </span>:port 50655</span>} </span>;; customize the nrepl session that shadow-cljs will run 
</span>
</span> </span>:builds </span>{</span>:game
</span>          {</span>:target :browser
</span>           </span>:output-dir </span>"</span>public/js</span>"
</span>           </span>:asset-path </span>"</span>/js</span>"
</span>           </span>:modules </span>{</span>:main </span>{</span>:init-fn</span> mazeboard.ui.core/init}
</span>                     </span>:portfolio </span>{</span>:init-fn</span> mazeboard.ui.portfolio/init
</span>                                 </span>:depends-on </span>#{</span>:main</span>}}}}}}
</span></code></pre>
Generally speaking this is just a common shadow-cljs config, and you should rely on
the (extensive) shadow-cljs docs to create your own, but there are a couple
of tweaks that are worth mentioning in the context of this app:</p>

dev-http</code> key: Portfolio has its own resources that must be included to render its interface</li>
nrepl</code> key: I like to run the nrepl process out of the IDE so that I connect to it with different editors/IDEs (and survive IDE life cycle if needed), I include cider to make it work with Emacs but it is not strictly needed for other editors/IDEs</li>
modules</code> key: each key in this map will generate a new JS file; there is one module for the game and one for portfolio, each with its own specific init fn</li>
</ul>
I have setup two pages (or views), one for the game and for portfolio, the first one will serve
the game and the other one (guess what?) portfolio UI. Given the provided build settings we will
have main.js</code> for our app code and portfolio.js</code> for portfolio; you want to have an html file
embedding main.js</code>for the main app and another embedding portfolio.js</code> for (well...) the
Portfolio UI. Given that the portfolio</code> key in modules</code> depends on main</code> module the custom
view (or html) for Portfolio must also include main</code> to be able to access your app's components, 
something like:</p>
<!</span>DOCTYPE </span>html</span>>
</span><</span>html </span>lang</span>="</span>en</span>">
</span>  <</span>head</span>>
</span>    <</span>title</span>>Portfolio Canvas</</span>title</span>>
</span>  </</span>head</span>>
</span>  <</span>body</span>>
</span>    <</span>main </span>id</span>="</span>canvas</span>" </span>role</span>="</span>main</span>"></</span>main</span>>
</span>    <</span>script </span>src</span>="</span>/js/main.js</span>"></</span>script</span>>
</span>    <</span>script </span>src</span>="</span>/js/portfolio.js</span>"></</span>script</span>>
</span>  </</span>body</span>>
</span></</span>html</span>>
</span></code></pre>
With this setup it is possible to access the game code at the root of the dev webserver
and preview components at /portfolio.html</code> for quick dev and tests.</p>
It is also worth mentioning that Portfolio UI can be customized quite a bit, for now my
portfolio namespace is as as simple as referencing the scenes to render and start the UI
adding my custom CSS to render my components:</p>
(</span>ns</span> mazeboard.ui.portfolio
</span>  (</span>:require </span>[portfolio.ui </span>:as</span> ui]
</span>            [mazeboard.ui.scenes.game-scene]
</span>            [mazeboard.ui.scenes.profile-scene]))
</span>
</span>::mazeboard.ui.scenes.game-scene
</span>::mazeboard.ui.scenes.profile-scene
</span>
</span>(</span>defn </span>init
</span>  []
</span>  (</span>ui/start! </span>{</span>:config </span>{</span>:css-paths </span>["</span>/css/style.css</span>"]}}))
</span></code></pre>
Update 2023-11-18 12:30 CET</em></p>
Thomas Heller pointed out in a comment</a>
on Reddit that the shadow-cljs build settings are not ideal because:</p>

portfolio would be included in the release build</li>
portfolio module depends on game module and this forces including both in portfolio.html and also would run the init-fn</code> of both modules</li>
</ul>
A new version of the :builds</code> map is the following</p>
{</span>:game
</span> {</span>:target :browser
</span>  </span>:output-dir </span>"</span>public/js</span>"
</span>  </span>:asset-path </span>"</span>/js</span>"
</span>  </span>:compiler-options </span>{</span>:infer-externs :auto :output-feature-set :es6</span>}
</span>  </span>:modules </span>{</span>:main </span>{</span>:init-fn</span> mazeboard.ui.core/init}}}
</span> 
</span> </span>:portfolio
</span> {</span>:target :browser
</span>  </span>:output-dir </span>"</span>public/portfolio-js</span>"
</span>  </span>:asset-path </span>"</span>/portfolio-js</span>"
</span>  </span>:compiler-options </span>{</span>:infer-externs :auto :output-feature-set :es6</span>}
</span>  </span>:modules </span>{</span>:main </span>{</span>:init-fn</span> mazeboard.ui.portfolio/init}}}}
</span></code></pre>
Now to run/watch both the game and portfolio we have to run</p>
$ shadow-cljs watch game portfolio</code></p>
At this point it will be possible to access the game at http://localhost:8080 and
portfolio at http://localhost:8080/portfolio.html. This is how the game and portfolio will
look when the build will finish (sorry for the terrible graphics...)</p>
Game
</p>
Portfolio rendering a board row
</p>
Portfolio rendering the full board
</p>
First impressions</h1>
The first objective of this reboot was to setup a frontend first development environment,
focusing on getting a view of the game board and preparing the work for future development.
So far I am quite satisfied with the result, being able to develop components in isolation
and preview them both in Portfolio and the app.</p>
Future work</h1>
Next step would be to setup a playable version of the game, loosely based on the approach
show in Parens of the dead but with a twist: instead of setting up a backend I am planning
to simulate the command/event based approach solely in the frontend for now, putting the logic
in cljc files so that, when it will come the time to have a backend, I can reuse most of it.
So far the dev experience has been great and I am hyped to work on and see the results of the
next steps!</p>
Sources of this project can be found here</a>.</p>
Mazeboard related posts</h1>

Mazeboard 0</a> - Project intro</li>
Mazeboard 1</a> - Dumdom event handler</li>
Mazeboard 2</a> - Using core async to decouple UI and game logic</li>
Mazeboard 3</a> - Improving UI and game logic decoupling</li>
Mazeboard 4</a> - Replacing commands with events</li>
Mazeboard 5</a> - More on actions, adding tests and future multiplayer architecture</li>
</ul>


Checkpoint 2023-11-01
Unknown — Wed, 01 Nov 2023 00:00:00 +0000
[2023-11-01] What's up?</h1>
Hello everyone, long time no see!</p>
After a break I've finally found new topics to work on and write about but,
before jumping to the upcoming plans lets take a look of what has happened
meanwhile.</p>
Unrefined</h1>
Unrefined</a> is the closest thing to a product that I have developed in the 
past year, it is currently used at $WORK and I have tried to understand if
it could be valuable to other teams. The feedback I've received was not great, the
most positive comments were "interesting" and "I see the value proposition" but 
it haven't move past that and, worse, haven't reached the point to start a trial with
the teams I've been in contact with. At least I've had a lot of fun working on it and
learned a lot in the process, so the glass is still half full ;)</p>
Unfortunately I've realized that to move it forward it would require a non trivial
effort compared to the potential revenue I can expect from it, based on the feedback
I have received by potential users. It will be on maintenance mode for my only user,
and if someone else wants to try it, it is still free (as in beer and in speech).</p>
Clojure Bites</h1>
The series covered simple topics, mostly targeting Clojure beginners like
myself and usually centered on the work I was doing on Unrefined. After few posts,
I've had an hard time finding interesting topics to cover, which is reasonable given
that I was not progressing with Unrefined and writing about something not directly
useful was feeling a bit too artificial. If I have not experienced a problem that I
wanted to solve I find it hard to give an honest perspective on the solution that I 
am covering.</p>
This does not mean I haven't tried, this is a list of topics I had (have) in the
backlog, loosely based on my own requirements for Unrefined, but not limited to it:</p>

Using Keycloak as an identity provider for Unrefined.</li>
Enabling custom estimation cheat sheets in Unrefined...which required a lot of frontend work I was not ready for.</li>
Building on the previous point I spent some time evaluating what options are available was in the frontend space, figwheel-main VS shadow-cljs, reagent VS rum VS helix/UIx2.</li>
Creating a service to simplify Server Sent Events solutions that scale; based on what I've seen in Unrefined, it might be useful to rely on a third party solution to handle SSE traffic.</li>
Using Clerk notebooks to work on support tickets at work as a way to document the process and create a common code base.</li>
Using Datascript as the state of an Entity Component System library for ClojureScript; this looks extremely exciting and I had a couple of games I can adapt to this approach.</li>
Using what I've learned from Unrefined to, finally, move one simple game (Mazeboard, ore on this later) to the next step, enabling realtime multiplayer games and experiment with its logic.</li>
</ul>
So yeah, so many topics at hand but I did not focus on any of them, and I was stuck (and it sucks!).</p>
Next steps</h1>
Taking Unrefined out of the equation is making it easier to decide where to spend
my time, optimizing for having fun while learning something, and the winner is:
revisiting Mazeboard's code base to finally have a way to test its gameplay, with real players, building on top of what I've learned in the past months.</p>
This will cover so many topics, possibly creating material for posts targeting people
with different levels of experience in Clojure and its tools/libraries, for example:</p>

Setting up a Shadow CLJS</a> project</li>
Introducing a frontend library (or two just to show different approaches, I am looking at dumdom</a> now)</li>
Working on UI components and previewing them with portfolio</a></li>
Live multiplayer game updates using SSE</li>
Persisting the game state (probably with some sort of Datalog DB)</li>
</ul>
I think these are reasonaly simple topics to get started again, fueled by the future
work on Mazeboard. After this step, it will be possible to focus on broader topics,
like identity management systems, scaling, logging and monitoring and who knows what.
At least at this point the basics will be covered and experience should suggest
what/where to look at.</p>
The plan look very interesting (at least for me) and doable, and knowing that I am
back again at experimentation VS building a product (that no one needs, anyway) is
quite exciting!
This is my sweet spot, happy to be back at it!</p>
Mazeboard</h1>
The game should be a rogue like board game where player explore a maze (I haven't
decided on theme yet, a dungeon? An alien spaceship?), trying to get a treasure and
bring it back at the starting position, other players can move towards the goal or
prevent other to win; it should be possible to print the board pieces and play with
a normal dice or a coin (I still have the paper prototype somewhere...). It is not a
surprise that it looks boring, I've had the idea during a boring meeting after all! :)
Maybe it will turn out as a playable but I will never know until someone will play
with it; relying on local, in person tests, is for sure out the
equation, so online game it is!</p>
Few years ago my go-to approach was to build the logic on a sorts of backend, just to
abandon the project when it came the time for the user facing part. This is silly!
Now I've leaned that having a user facing prototype is much, much more valuable, so
I am revisiting Mazeboard's code base to quickly provide a way to play it, favoring
user facing work.</p>
Sources will be available here</a>, I am halfway
on re-doing the frontend part, at least the UI components, and I have started using
dumdom + portfolio; excluding few headaches when setting everything up, so far the
experience have been quite positive!</p>
Conclusion</h1>
Finally I am back again at the whiteboard, experimenting with ideas and libraries,
gaining experience and hopefully giving back in form of posts or live coding sessions.
(quite unlikely but I'd like to try again! ;) )</p>


Clojure bites - Profiling with Tufte
FPSD — Thu, 31 Aug 2023 00:00:00 +0000
Overview</h1>
If we have a system in production it is quite common to want
to understand how many resources it is using as a whole and for
each of its components; this information can be used to plan
how much hardware we have to provision or to optimize the costs,
identifying expensive "code paths" and planning some work to
optimize its operational costs. Another use of this information
is resource monitoring, finding performance regressions after
we have deployed some changes to our systems.</p>
To collect this information we can use tufte</a>, a profiling library, and
to send collected metrics to a event/log collector we can use
mulog</a> which makes it easy to log data instead of text messages that
we would need to parse later. (see HERE</a> for an introduction to mulog)
In this post we are going to use Opensearch</a> but the same approach
can be easily ported to other tools or even IaaS/PaaS.</p>
The plan</h1>
There is a lot to do, so better to lay out a plan, it is not important
if it is not super well defined and should work to help me to not go out
of the track:</p>

Introduce tufte and apply it to some sample code.</li>
Create an api or webpage using ring + reitit that uses the code that we are profiling.</li>
Add a middleware to the router to log with mulog, writing to file.</li>
Collect metrics and log them periodically.</li>
Setup Opensearch and log to it.</li>
Show logs and potentially a dashboard.</li>
</ul>
Introducing Tufte</h1>
Taking from the project's README</a>:</p>

Tufte allows you to easily monitor the ongoing performance of your Clojure and ClojureScript applications in production and other environments.</p>
It provides sensible application-level metrics, and gives them to you as Clojure data that can be easily analyzed programmatically.</p>
</blockquote>
I don't think there is a better way to describe it so I will not try.</p>
No, ok, I will try. In our context it can be seen as a tool to collect
metrics about our system, at runtime, with low overhead, and to work on
them (the metrics) as pure data to do, well, whatever we want to, for
example logging that data somewhere to be used later or directly fire
alarms if some part of the system is not behaving as expected.</p>
This looks interesting in theory but how does it look in practice?</p>
First of all we have to add tufte to our dependencies, as of today
the current version is 2.6.0</code>, so we can add the dependency like so:</p>
  {</span>:deps </span>{com.taoensso/tufte {</span>:mvn/version </span>"</span>2.6.0</span>"}}}
</span></code></pre>
Now we can run a tiny experiment to get a taste of the data we can get;
first of all lets pretend we want to profile the good old FizzBuzz, for
which we have two implementations:</p>
  (</span>ns</span> codes.fpsd.fizzbuzz
</span>    (</span>:require </span>[clojure.core.match </span>:refer </span>[match]]))
</span>  
</span>  (</span>defn </span>fizz-buzz-match
</span>    [n]
</span>    (</span>match </span>[(</span>mod</span> n </span>3</span>) (</span>mod</span> n </span>5</span>)]
</span>           [</span>0 0</span>] "</span>FizzBuzz</span>"
</span>           [</span>0</span> _] "</span>Fizz</span>"
</span>           [_ </span>0</span>] "</span>Buzz</span>"
</span>           </span>:else</span> n))
</span>  
</span>  (</span>defn </span>fizz-buzz-cond
</span>    [n]
</span>    (</span>let </span>[m3 (</span>mod</span> n </span>3</span>)
</span>          m5 (</span>mod</span> n </span>5</span>)]
</span>      (</span>cond
</span>        (</span>and</span> m3 m5) "</span>FizzBuzz</span>"
</span>        m3 "</span>Fizz</span>"
</span>        m5 "</span>Buzz</span>"
</span>        </span>:else</span> n)))
</span></code></pre>
Now we can profile it:</p>
  (</span>ns</span> codes.fpsd.perf-log-es
</span>    (</span>:require </span>[taoensso.tufte </span>:as</span> t]
</span>              [codes.fpsd.fizzbuzz </span>:as</span> fb]))
</span>  
</span>  </span>;; Request to send `profile` stats to `println`:
</span>  (</span>t/add-basic-println-handler! </span>{})
</span>  
</span>  (</span>t/profile
</span>    {}
</span>    (</span>dotimes </span>[_ </span>50</span>]
</span>      (</span>t/p </span>:fizz-buzz-match </span>(</span>fb/fizz-buzz </span>10000</span>))
</span>  
</span>      (</span>t/p </span>:fizz-buzz-cond </span>(</span>fb/fizz-buzz-cond </span>10000</span>))))
</span></code></pre>
Which will output something like:</p>
pId                  nCalls        Min      50% ≤      90% ≤      95% ≤      99% ≤        Max       Mean   MAD      Clock  Total
</span>
</span>:fizz-buzz-match         50     9.01μs     9.46μs    12.19μs    16.52μs   165.84μs   305.55μs    16.47μs  ±71%   823.61μs    31%
</span>:fizz-buzz-cond          50     8.08μs     8.48μs    12.03μs    13.02μs    84.26μs   150.11μs    12.41μs  ±46%   620.52μs    23%
</span>
</span>Accounted                                                                                                          1.44ms    55%
</span>Clock                                                                                                              2.65ms   100%
</span></code></pre>
Lets break down what we did and what we got:</p>

We want to profile some code, in this case two FizzBuzz implementations</li>
We have told tufte to print its results; we will see later other way to get the numbers</li>
We have profiled the two functions</li>
And finally we got our results in a nicely formatted table which gives us an idea of what we can get from tufte</li>
</ul>
One thing to notice is that tufte only profiles CPU time and does not take
into account memory usage; depending on our requirements it may be a deal
breaker for needs, in that case we can track memory usage on the side with
other tools or replace it completely.</p>
Using the println handler may be enough when profiling something locally but
for something running in a server (or cluster) it may be more convenient to
log the metrics or apply some logic to collected metrics. For these use case
there are two options:</p>

tufte/profiled</li>
tufte/add-accumulating-handler!</li>
</ul>
profiled</h2>
Like profile, it will run the enclosed forms but instead of directly returning
the result of the form and send the pstats to the configured handler, it returns
a tuple composed by:</p>

the value of the form</li>
a pstats object that can be inspected to collect the collected metrics</li>
</ul>
To have a better understanding of the output of profiled we can run the following form:</p>
  (</span>def </span>res </span>(</span>t/profiled
</span>               {}
</span>             (</span>dotimes </span>[_ </span>50</span>]
</span>               (</span>t/p </span>:fizz-buzz-match </span>(</span>fb/fizz-buzz </span>10000</span>))
</span>  
</span>               (</span>t/p </span>:fizz-buzz-cond </span>(</span>fb/fizz-buzz-cond </span>10000</span>)))))
</span>  
</span>  (</span>nth</span> res </span>0</span>) </span>;; => nil
</span>  (</span>deref </span>(</span>nth</span> res </span>1</span>)) </span>;; => continues...
</span>  
</span>  {</span>:clock </span>{</span>:t0 30597219635504</span>, </span>:t1 30597220674063</span>, </span>:total 1038559</span>}</span>,
</span>   </span>:stats </span>{</span>:fizz-buzz-match </span>{</span>:min 3973</span>, </span>:mean 5973.8</span>, </span>:p75 4196.75</span>, </span>:mad-sum 165725.20000000007</span>, </span>:p99 47871.74999999987</span>, </span>:n 50</span>, </span>:p25 4052.0</span>, </span>:p90 4959.9000000000015</span>, </span>:var 1.1782216812E8</span>, </span>:max 81400</span>, </span>:mad 3314.5040000000013</span>,
</span>                             </span>:loc </span>{</span>:ns </span>"</span>codes.fpsd.perf-log-es</span>"</span>, </span>:line 20</span>, </span>:column 14</span>, </span>:file </span>"</span>/home/foca/work/playground/src/codes/fpsd/perf_log_es.clj</span>"}</span>,
</span>                             </span>:last 81400</span>, </span>:p50 4116.5</span>, </span>:sum 298690</span>, </span>:p95 6182.649999999998</span>, </span>:var-sum 5.891108406E9</span>}</span>,
</span>           </span>:fizz-buzz-cond </span>{</span>:min 3630</span>, </span>:mean 4549.58</span>, </span>:p75 3795.0</span>, </span>:mad-sum 71821.36000000004</span>, </span>:p99 14535.559999999994</span>, </span>:n 50</span>, </span>:p25 3707.0</span>, </span>:p90 4196.1</span>, </span>:var 7170650.4836</span>, </span>:max 15690</span>, </span>:mad 1436.427200000001</span>,
</span>                            </span>:loc </span>{</span>:ns </span>"</span>codes.fpsd.perf-log-es</span>"</span>, </span>:line 22</span>, </span>:column 14</span>, </span>:file </span>"</span>/home/foca/work/playground/src/codes/fpsd/perf_log_es.clj</span>"}</span>,
</span>                            </span>:last 15690</span>, </span>:p50 3735.5</span>, </span>:sum 227479</span>, </span>:p95 12587.149999999998</span>, </span>:var-sum 3.5853252418E8</span>}}
</span>   }
</span></code></pre>
WOW, Now we are speaking! Look at that, we have a "raw" representation of what the
println handler showed us but now we can work with it! For example we could directly
rise alerts depending on some collected metric or send the whole or a sample of that
data to a log collector. A recap of what we did and what we've got:</p>

we have profiled a form with profiled instead of profile</li>
we've got a tuple with the result of the form and the pstat object which contains</li>
total amount of time of the enclosed forms</li>
details of each p form, including percentile stas and the location of the code generating them</li>
</ul>
Inspecting pstats directly as shown in the previous example may not be the most
convenient way to get value out of it, especially in a development environment; one
handy way of collecting this data while working on some piece of code and profiling it,
is to send pstats to Portal</a> and use its powerful
views to explore the data generated by tufte, here is an example view:</p>
</p>
There are multiple ways to get tufte's pstats to portal but the simplest form
"to get there" is something like:</p>
  (</span>require </span>'[portal.api </span>:as</span> portal])
</span>  (</span>def </span>p </span>(</span>portal/open</span>))
</span>  
</span>  (</span>add-tap </span>#'portal/submit)
</span>  
</span>  (</span>let </span>[[res pstats] (</span>t/profiled
</span>                       {}
</span>                       (</span>dotimes </span>[_ </span>50</span>]
</span>                         (</span>t/p </span>:fizz-buzz-match </span>(</span>fb/fizz-buzz </span>10000</span>))
</span>  
</span>                         (</span>t/p </span>:fizz-buzz-cond </span>(</span>fb/fizz-buzz-cond </span>10000</span>))))]
</span>    (</span>tap></span> res)
</span>    (</span>tap> </span>@pstats))
</span></code></pre>
Portal's docs</a> have a section
to work with tufte available here</a>.</p>
add-accumulating-handler!</h2>
It is not always ideal, or needed to consume pstats objects on the fly,
even if it is cheap it will still adds up to your business logic runtime
or endpoints' response times. To avoid this we can accumulate pstats and
consume that on demand or periodically (for example every minute).
tufte/add-accumulating-handler!</code> will create a new pstats accumulator, returning it,
which can later be drained to get all collected stats to do, well, whatever we
want to, for example to log the accumulated stats somewhere. De-refing an accumulator
will return a map in the form of</p>
  {</span>:profile-id</span> pstat}
</span></code></pre>
Like the pstat object returned by profiled</code>, to access its content we must deref</code> it; here is a simple example:</p>
    (</span>def </span>accumulated-stats </span>(</span>t/add-accumulating-handler! </span>{}))
</span>    
</span>    (</span>t/profile
</span>     {</span>:id :testing-1</span>}
</span>     (</span>dotimes </span>[_ </span>50</span>]
</span>       (</span>t/p </span>:fizz-buzz-match </span>(</span>fb/fizz-buzz </span>10000</span>))
</span>       (</span>t/p </span>:fizz-buzz-cond </span>(</span>fb/fizz-buzz-cond </span>10000</span>))))
</span>    
</span>    (</span>t/profile
</span>     {</span>:id :testing-2</span>}
</span>     (</span>dotimes </span>[_ </span>50</span>]
</span>       (</span>t/p </span>:fizz-buzz-match </span>(</span>fb/fizz-buzz </span>20000</span>))
</span>       (</span>t/p </span>:fizz-buzz-cond </span>(</span>fb/fizz-buzz-cond </span>20000</span>))))
</span>    
</span>    (</span>doseq </span>[[p-id pstats] @accumulated-stats]
</span>      (</span>tap> </span>[p-id @pstats]))
</span></code></pre>
And here is how it looks in Portal:</p>

</p>
Configuring Tufte</h2>
It is worth spending few words on how we can configure Tufte to fine
tune it and to adapt it to our use case. We may have noticed that, profile,
profiled and add-accumulating-handler! accept a configuration map which,
excluding the last example, we left empty for simplicity. Here is a breakdown
of the available configuration options.</p>
profile, profiled, p:</p>

:level</code>: Optional profiling level, defaults to 5</li>
:when</code>: Optional arbitrary conditional form that takes no arguments and return truty to enable profiling, falsy otherwise</li>
:id</code>: Optional identifier provider to handlers (print or custom)</li>
:data</code>: Optional custom data to be provided to handlers</li>
</ul>
To be noted that the profiling level can be customized at runtime
by altering the var tufte/*min-level*</code>, the JVM property taoensso.tufte.min-level</code>
or the environment variable TAOENSSO_TUFTE_MIN_LEVEL</code>.</p>
Another configuration that will filter these forms is tufte/*ns-filter*</code> which can
customized by using alter-var-root</code>, or JVM property taoensso.tufte.ns-pattern</code> or
the environment variable TAOENSSO_TUFTE_NS_PATTERN</code>.</p>
add-accumulating-handler!</p>

:ns-pattern</code>: same as the global tufte/*ns-filter*</code> but as per handler basis, useful to have different handlers for ns patterns</li>
</ul>
Refer to the docs</a> to have the full picture about the configuring options.</p>
Comparing the two options</h2>
profiled will return pstats to be used right away but we pay the price for each call
to do something with the returned pstats, instead, add-accumulating-handler! will send
accumulated pstats to an handler that will be drained at convenient times, we can think
of it like an out of band process.
It is nice to be able to have both behaviors available so that we can fine tune our profiling
setup based on environments, namespace or custom conditions.</p>
Small section about collecting data and log it with mulog</h1>
We have briefly seen how we can profile our code with tufte and get some data about it,
we have also sent the data to portal which helps us to visualize it. This setup can be
handy while developing however, we are still far away from an actionable setup where we can
create or dashboards using the collected data. The missing pieces are:</p>

something to send the collected data "somewhere", for example mulog</li>
a "somewhere" that can ingest this data so that we can work with it, for example opensearch</li>
</ul>
Sending pstats to a log collector</h1>
Next sections are about setting Opensearch and mulog as a way to send profiling data
"somewhere" else. If you are already familiar with these topics you can skip these
details (or maybe read it anyway and help me to fix mistakes ;) ).</p>
Opensearch</h2>
To simplify (quite a lot) we can setup a local single node opensearch, available at port 9200,
with its nice dashboard, available at port 5601 accessible with admin:admin crendentials, using
docker compose and the following docker-compose.yml:</p>
    </span>services</span>:
</span>      </span>os-node1</span>:
</span>        </span>image</span>: </span>opensearchproject/opensearch:2.3.0
</span>        </span>container_name</span>: </span>os-node1
</span>        </span>environment</span>:
</span>          - </span>cluster.name=os-cluster
</span>          - </span>node.name=os-node1
</span>          - </span>discovery.type=single-node
</span>          - </span>bootstrap.memory_lock=true </span># along with the memlock settings below, disables swapping
</span>          - "</span>OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m</span>" </span># minimum and maximum Java heap size, recommend setting both to 50% of system RAM
</span>        </span>ulimits</span>:
</span>          </span>memlock</span>:
</span>            </span>soft</span>: </span>-1
</span>            </span>hard</span>: </span>-1
</span>          </span>nofile</span>:
</span>            </span>soft</span>: </span>65536 </span># maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
</span>            </span>hard</span>: </span>65536
</span>        </span>volumes</span>:
</span>          - </span>os-data1:/usr/share/opensearch/data
</span>        </span>ports</span>:
</span>          - </span>9200:9200
</span>          - </span>9600:9600 </span># required for Performance Analyzer
</span>        </span>networks</span>:
</span>          - </span>os-net
</span>      </span>os-dashboards</span>:
</span>        </span>image</span>: </span>opensearchproject/opensearch-dashboards:2.3.0
</span>        </span>container_name</span>: </span>os-dashboards
</span>        </span>ports</span>:
</span>          - </span>5601:5601
</span>        </span>expose</span>:
</span>          - "</span>5601</span>"
</span>        </span>environment</span>:
</span>          </span>OPENSEARCH_HOSTS</span>: '</span>["https://os-node1:9200"]</span>'
</span>        </span>networks</span>:
</span>          - </span>os-net
</span>    
</span>    </span>volumes</span>:
</span>      </span>os-data1</span>:
</span>    
</span>    </span>networks</span>:
</span>      </span>os-net</span>:
</span></code></pre>
If we run this we can send our logs to localhost:9200</code> and point the browser to http:localhost:5601</code>
to see logs and setup our dashboards and alerts. One litle extra step is needed
to be able to send logs to this dockerized opensearch setup. Given that it uses a self signed
certificate we must instruct mulog to ignore certificate validation errors and also
we must provide our credentials (admin:admin), this is easily done by adding the
following map to mulog's configuration:</p>
  </span>;; extra http options to be forwarded to the HTTP client
</span>  </span>:http-opts </span>{</span>:insecure? true
</span>              </span>:basic-auth </span>["</span>admin</span>" "</span>admin</span>"]}
</span></code></pre>
mulog uses the clj-http</a> HTTP client library
so any setting provided in the :http-opts will be forwarded to that library when
making HTTP calls to send data to opensearch. These settings are neede because
the dockerized opensearch runs on HTTPS with a self signed cert (:insecure?</code>)
and with basic auth credentials (:basic-auth</code>); this is not a production grade
setup but is good enough to test locally.</p>
mulog</h2>
mulog can help to send metrics to a collector and it has quite a lot of available publishers
that can send logs to different type of systems like ElasticSearch, Opensearch and many others
(see here</a> for a full list).</p>
For the purpose of this article lets focus on Opensearch for which can use the ElasticSearch
publisher out of the box.</p>
The bare minimum configuration needed to send events to Opensearch, considering our dockerized
environment, is the following:</p>
{</span>:mulog/publisher </span>{
</span>  </span>:type :elasticsearch
</span>  </span>;; Elasticsearch endpoint (REQUIRED)
</span>  </span>:url  </span>"</span>https://localhost:9200/</span>"
</span>
</span>  </span>;; extra http options to pass to the HTTP client
</span>  </span>:http-opts </span>{</span>:insecure? true
</span>              </span>:basic-auth </span>["</span>admin</span>" "</span>admin</span>"]}
</span>}}
</span></code></pre>
This configuration will be the argument of com.brunobonacci.mulog/start-publisher!</code>. To be 
able to use this publisher we must also add one more dependency com.brunobonacci/mulog-elasticsearch</code>
to our project.clj or deps.end.</p>
We can start the publisher in our preferred way, manually or using an application
state management library like
integrant</a>, component</a>,
mount</a> and so on.</p>
Putting it all together</h1>
So far we have seen:</p>

How to profile our code</li>
How to get pstats in different ways, on demando or accumulating it</li>
How to visualize that data</li>
</ul>
Now, finally, we can put it all together in a tiny example web app that sends the pstats to
Opensearch. I'll try to keep it as simple as possible, to capture the important details.</p>
;; web server setup with profiling middleware 
</span>(</span>defn </span>request->endpoint-name
</span>  [request]
</span>  (</span>-></span> request </span>:reitit.core/match :data </span>(</span>get </span>(</span>:request-method</span> request)) </span>:name</span>))
</span>
</span>(</span>defn </span>profiler-middleware
</span>  "</span>Middleware that wraps handlers with a call to t/profile
</span>  using the endpoint name as the id of the </span>"
</span>  [wrapped]
</span>  (</span>fn </span>[request]
</span>    (</span>t/profile </span>{</span>:id :ring-handler</span>}
</span>               (</span>t/p </span>(</span>or </span>(</span>request->endpoint-name</span> request) </span>:unnamed-handler</span>)
</span>                    (</span>wrapped</span> request)))))
</span>
</span>(</span>defn </span>fizz-buzz-handler
</span>  "</span>Ring handler that takes the number from the request path-params
</span>  and returns a JSON with the resulf of calling fizz-buzz</span>"
</span>  [request]
</span>  (</span>try
</span>    (</span>let </span>[number (</span>-></span> request
</span>                     </span>:path-params
</span>                     </span>:number
</span>                     Integer/parseInt)]
</span>      (</span>u/log </span>:fizz-buzz-handler :argument</span> number)
</span>      {</span>:status 200
</span>       </span>:body </span>(</span>json/generate-string </span>{</span>:result </span>(</span>for </span>[n (</span>range </span>1 </span>(</span>inc</span> number))]
</span>                                              (</span>fb/fizz-buzz</span> n))})
</span>       </span>:content-type </span>"</span>application/json</span>"})
</span>    (</span>catch</span> NumberFormatException e
</span>      (</span>u/log </span>:fizz-buzz-handler :error </span>(</span>str</span> e))
</span>      {</span>:status 400
</span>       </span>:body </span>(</span>json/generate-string </span>{</span>:error </span>(</span>str </span>"</span>Invalid numeric parameter </span>" e)})
</span>       </span>:content-type </span>"</span>application/json</span>"})))
</span>
</span>(</span>defn </span>create-app
</span>  "</span>Return a ring handler that will route /events to the SSE handler
</span>   and that will servr  static content form project's resource/public directory</span>"
</span>  []
</span>  (</span>ring/ring-handler
</span>   (</span>ring/router
</span>    [["</span>/fizz-buzz/:number</span>" {</span>:get </span>{</span>:handler</span> fizz-buzz-handler
</span>                                  </span>:name ::fizz-buzz</span>}}]
</span>     ]
</span>     
</span>    {</span>:data </span>{</span>:middleware </span>[profiler-middleware
</span>                         params/wrap-params]}})
</span>   ))
</span>
</span>;; define all integrant components
</span>
</span>(</span>defmethod </span>ig/init-key </span>:profiler/accumulator </span>[_ config]
</span>  (</span>t/add-accumulating-handler! </span>{}))
</span>
</span>(</span>defmethod </span>ig/init-key </span>:mulog/publisher </span>[_ config]
</span>  (</span>u/start-publisher!</span> config))
</span>
</span>(</span>defmethod </span>ig/init-key </span>:profiler/drain </span>[_ {</span>:keys </span>[delay accumulator]}]
</span>  (</span>future
</span>    (</span>while </span>true
</span>      (</span>Thread/sleep</span> delay)
</span>      </span>;; drain pstats if any
</span>      (</span>when-let </span>[stats @accumulator]
</span>        (</span>doseq </span>[[p-id pstats] stats]
</span>          (</span>doseq </span>[[handler handler-stats] (</span>:stats </span>@pstats)]
</span>            (</span>let </span>[{</span>:keys </span>[loc min max mad p95 p99]} handler-stats]
</span>              (</span>u/log </span>:accumulator-drain :perf-id</span> p-id </span>:handler</span> handler)
</span>              (</span>u/log</span> p-id </span>:handler</span> handler </span>:loc</span> loc </span>:min</span> min </span>:max</span> max </span>:mad</span> mad </span>:p95</span> p95 </span>:p99</span> p99))))))
</span>    (</span>u/log </span>:accumulator-drain-finished</span>)))
</span>            
</span>
</span>(</span>defmethod </span>ig/halt-key! </span>:profiler/drain </span>[_ pstat-future]
</span>  (</span>future-cancel</span> pstat-future))
</span>
</span>(</span>defmethod </span>ig/init-key </span>:http/server </span>[_ config]
</span>  (</span>http/start-server </span>(</span>create-app</span>) config))
</span>
</span>(</span>defmethod </span>ig/halt-key! </span>:http/server </span>[_ server]
</span>  (</span>.close</span> server))
</span>
</span>;; and the configuration for all components of the system
</span>;; being basically all "raw" data it can from an external
</span>;; source like an edn file
</span>(</span>def </span>config </span>{</span>:profiler/accumulator </span>{}
</span>             </span>:profiler/drain </span>{</span>:delay 5000
</span>                              </span>:accumulator </span>(</span>ig/ref </span>:profiler/accumulator</span>)}
</span>             </span>:mulog/publisher </span>{</span>:type :elasticsearch
</span>                               </span>;; Elasticsearch endpoint (REQUIRED)
</span>                               </span>:url  </span>"</span>https://localhost:9200/</span>"
</span>
</span>                               </span>;; extra http options to pass to the HTTP client
</span>                               </span>:http-opts </span>{</span>:insecure? true
</span>                                           </span>:basic-auth </span>["</span>admin</span>" "</span>admin</span>"]}}
</span>             
</span>             </span>:http/server </span>{</span>:port 8080
</span>                           </span>:join? false</span>}})
</span>
</span>;; prepare the system, it will be intilialized only after
</span>;; de-referencing the system var
</span>(</span>def </span>system
</span>  (</span>delay </span>(</span>ig/init</span> config)))
</span></code></pre>
It is quite a lot of code! Lets break it down:</p>

We have setup a small web app with one endpoint that will return a list of the application of fizz-buzz</code> to numbers from 1 to :number</code></li>
We have defined a middlware profiler-middleware</code> that will wrap any handler and will profile it</li>
We spawn a thread that that, at defined times, will drain the stats accumulator and will log some stats using mulog</code> targetting opensearch</code></li>
Everything is setup using integrant</a></li>
</ul>
If we start the system we can test our new shiny API usung curl</code>:</p>
❯</span> curl http://localhost:8080/fizz-buzz/foo
</span>{</span>"</span>error</span>"</span>:</span>"</span>Invalid numeric parameter java.lang.NumberFormatException: For input string: </span>\"</span>foo</span>\"</span>"}
</span>
</span>❯</span> curl http://localhost:8080/fizz-buzz/10
</span>{</span>"</span>result</span>"</span>:[1,2,</span>"</span>Fizz</span>"</span>,4,</span>"</span>Buzz</span>"</span>,</span>"</span>Fizz</span>"</span>,7,8,</span>"</span>Fizz</span>"</span>,</span>"</span>Buzz</span>"</span>]</span>}
</span></code></pre>
After poking the endponits for a bit we should accumulated some stats and we should be able to 
look at opensearch dashboard to inspect logs and potentially create views and alerts, here is an example:</p>
</p>
The code supporting this post can be found here</a>.</p>
Conclusions</h1>
In this post we have seen how we can collect informations about our running system
and send them to a convenient place for subsequent analysis, to setup alerts or 
created beautiful visualizations. I did skip soe details, for example how to
configure integrant, mulog or opensearch and I suggest to refer to their own documentation.</p>
As a side note I think this post was a bit too big for a Clojure Bite issue, touching too
many arguments without focusing on one topic specifically. But I was already too late
when I've discovered this problem so the post it is what it is :D</p>
Discuss</h1>

Reddit</a></li>
Twitter</a></li>
Mastodon</a></li>
Linkedin</a></li>
</ul>
Updates</h1>
From comments in Reddit I've learnead about alternatives for profiling and tracing</p>
clj-async-profiler (comment</a>)</h2>
clj-async-profiler</a>, a much more
accurate tracing profiler; probably best suited when developing and not in production, but
still one more good tool to keep in mind.</p>
Edit:</p>
Looking at the README it looks like it is suitable for production environments too:</p>

During profiling, clj-async-profiler has very low overhead, so it is suitable for usage even in highly loaded production scenarios.</p>
</blockquote>
mulog (comment</a>)</h2>
mulog itself can create traces (using u/trace) for more fine grained metrics;
the tradoff is that the number of events to be recorded can be very high and
tufte's accumulated stats can mitigate this problem. Both tufte and mulog support
conditional profiling/logging (for example via sampling) which can help to keep
the number of events under control.</p>


Clojure bites - SSE with Aleph and Reitit
FPSD — Thu, 03 Aug 2023 00:00:00 +0000
Overview</h1>
To coordinate the refinement and estimation sessions in Unrefined
I wanted to have a way to give users instant feedback on some events
like people estimating or moving to the next ticket. At the same time
I wanted to keep the frontend code as simple as possible i.e. I tried
to avoid writing a SPA as much as possible so I settled to the following
setup:</p>

plain HTML GET for server side rendering of a view</li>
plain HTML POST for "actions" and server side rendering of the result</li>
Server Sent Events (SSE</a>) for realtime push updates</li>
</ul>
This setup worked pretty well, it is easy to maintain and requires not
much JavaScript code given the small size of this webapp.</p>
In this post I'll describe the basic building blocks to replicate such
setup using Aleph</a> which provides a nice implementation of SSE
(and Websockets, but not covered here) leveraging Manifold</a>; the same
result can be achieved using other web servers and core.async but I haven't tried it yet.</p>
SSE overview</h1>
Before jumping into the implementation it is better to spend few words on the
SSE spec. If you are already familiar with it you can jump to the next section,
or you can stay here and help me to spot and fix possible mistakes ;) .</p>
SSE provides a simple (but effective) way to stream events from a server to
subscribed clients. Clients create a persistent connection to a server and
wait for events being sent to them.</p>
The client connects to an endpoint using the EventSource</a> interface and the server
is expected to reply with `text/event-stream` MIME type, creating effectively
a persistent connection through which the server can send new events. See here</a>
for more details about the event format. As a quick introduction, events can be
just data:</p>
data: event payload
</span>
</span>data: event payload with
</span>data: multiple lines
</span>
</span></code></pre>
or event types can be included in the event itself:</p>
event: your-event-type
</span>deta: your event payload
</span>
</span></code></pre>
To recap:</p>

events are delimited by an empty line, keep this in mind when serializing your events</li>
events can have a type (or tag, however you want to call it) which can be handy when subscribing to specific events on the client side</li>
on first connection the endpoint MUST reply with the text/event-stream</code> MIME type</li>
</ul>
For more detailed information please refer to these links:</p>

MDN</a></li>
Spec</a></li>
W3C</a></li>
</ul>
There is a ton of material out there about SSE, these links are just quick references.</p>
Project setup</h1>
Time to get some code! Lets start with a simple base setup for the server side.</p>
We are going to create a web app with a single endpoint that will stream events
back to a client to get things started, later we will add a router to handle two
endpoints, one to send data and another to receive events and a tiny JS implementation
of the EventSource interface to render events to a page.</p>
We can start by putting down some deps, I am using tool.deps</code> but it should be easy
to port them to lain or boot if needed:</p>
    {</span>:deps </span>{org.clojure/clojure {</span>:mvn/version </span>"</span>1.11.1</span>"}
</span>            ring/ring-core {</span>:mvn/version </span>"</span>1.10.0</span>"}
</span>            aleph/aleph {</span>:mvn/version </span>"</span>0.6.3</span>"}}
</span></code></pre>
Now we can start with an HTTP handler, not the fanciest code you'll ever see in your
life but good enough to get started</p>
    (</span>ns</span> codes.fpsd.sse
</span>      (</span>:require </span>[aleph.http </span>:as</span> http]))
</span>    
</span>    (</span>defn </span>handler </span>[_req]
</span>      {</span>:status 200
</span>       </span>:headers </span>{"</span>content-type</span>" "</span>text/event-stream</span>"}
</span>       </span>:body </span>"</span>data: one event, k thanks bye!</span>\n</span>"})
</span>    
</span>    (</span>comment
</span>      (</span>http/start-server</span> handler {</span>:port 8080
</span>                                  </span>:join? false</span>})
</span>      </span>,</span>)
</span></code></pre>
After eveluating this code and the inner comment section we will have a running
web server that replies to every route with a text/event-source</code> body and will
close the connection afterwards; not too different compared to a "normal" HTTP
endpoint but we will get there. Here is the output of curl session:</p>
$</span> curl localhost:8080</span> -vv
</span>
</span>
</span># Trying 127.0.0.1:8080&#x2026;
</span>
</span>
</span># Connected to localhost (127.0.0.1) port 8080 (#0)
</span>
</span>> GET </span>/</span> HTTP/1.1
</span>> Host: </span>localhost:8080
</span>> User-Agent: </span>curl/7.81.0
</span>> Accept: </span>**/**
</span>> 
</span>
</span>
</span># Mark bundle as not supporting multiuse
</span>
</span>< HTTP/1.1 </span>200</span> OK
</span>< Content-Type: </span>text/event-stream
</span>< Server: </span>Aleph/0.6.3
</span>< Date: </span>Sun,</span> 30 Jul 2023 19:13:32 GMT
</span>< Connection: </span>Keep-Alive
</span>< content-length: </span>31
</span>< 
</span>data:</span> one event, k thanks bye!
</span>
</span>
</span># Connection #0 to host localhost left intact
</span></code></pre>
Streaming data</h1>
Aleph builds its functionalities on top of Manifold which provides asynchronous
data structures like manifold.deferred/deferred</code> or manifold.stream/stream</code>
(and more), these data structure can be used as the body of a response and
Aleph will return their contents as soon as they are available and will close
the connection as soon as these sources will be closed. Manifold has been released
almost at the same time of core.async</code> (docs</a>) and can be also used to work with async channels, which
can be handy if we have some part of our system that relies on them. Citing its
own Readme</a>:</p>
Manifold provides basic building blocks for asynchronous programming, and can be used as a translation layer between libraries which use similar, but incompatible, abstractions.
</span>
</span>Manifold provides two core abstractions: deferreds, which represent a single asynchronous value, and streams, which represent an ordered sequence of asynchronous values.
</span></code></pre>
Manifold API has a lot to offer and I encourage everyone to get to its docs
to get a better understanding of this library, to approach it quickly it could
be enough to see how it is used in Aleph's examples here</a>.
Other, more up to date examples are available here</a>.</p>
For the purpose of this exercise we can think of Manifold streams as channels to
which we can put data and from which Aleph read data to send back events to an
EventSource client.</p>
Lets re-write the handler in order to produce periodic events that later will be
consumed by a client</p>
    (</span>ns</span> codes.fpsd.sse
</span>      (</span>:require </span>[aleph.http </span>:as</span> http]
</span>                [manifold.stream </span>:as</span> s]))
</span>    
</span>    (</span>defn </span>format-event </span>[body]
</span>      (</span>str </span>"</span>data: </span>" body "</span>\n\n</span>"))
</span>    
</span>    (</span>defn </span>handler </span>[_req]
</span>      {</span>:status 200
</span>       </span>:headers </span>{"</span>content-type</span>" "</span>text/event-stream</span>"}
</span>       </span>:body </span>(</span>let </span>[counter (</span>atom </span>0</span>)]
</span>               (</span>s/periodically
</span>                </span>1000
</span>                #(</span>format-event </span>(</span>str </span>"</span>Sending event #</span>" (</span>swap!</span> counter inc)))))})
</span></code></pre>
This new endpoint will send an event every second, until the client closes the
connection; (manifold.stream/periodically delay-ms fn)</code> returns a stream to
which a new value is sent by calling fn</code> after delay-ms</code> milliseconds.
The format-event</code> fn is a helper that will return a SSE event, with only the
data</code> part, given any text renderable body</code>; it could be improved to properly
handle data bodies which include new lines generating multiple data:</code> section,
but I guess you can do that easily if you want to ;) .</p>
To recap the new endpoint will send back an event with a progressive number,
starting from 1, to every client that connects to the web server.</p>
This what we get if we try the endpoint with curl:</p>
$</span> curl localhost:8080</span> -vv
</span>
</span>
</span># Trying 127.0.0.1:8080&#x2026;
</span>
</span>
</span># Connected to localhost (127.0.0.1) port 8080 (#0)
</span>
</span>> GET </span>/</span> HTTP/1.1
</span>> Host: </span>localhost:8080
</span>> User-Agent: </span>curl/7.81.0
</span>> Accept: </span>**/**
</span>> 
</span>
</span>
</span># Mark bundle as not supporting multiuse
</span>
</span>< HTTP/1.1 </span>200</span> OK
</span>< Content-Type: </span>text/event-stream
</span>< Server: </span>Aleph/0.6.3
</span>< Date: </span>Mon,</span> 31 Jul 2023 17:37:15 GMT
</span>< Connection: </span>Keep-Alive
</span>< transfer-encoding: </span>chunked
</span>< 
</span>data:</span> Sending event </span>#1
</span>
</span>data:</span> Sending event </span>#2
</span>
</span>data:</span> Sending event </span>#3
</span>
</span>data:</span> Sending event </span>#4
</span>
</span>^C
</span></code></pre>
Oooook, finally something interesting is happening! A client connects to
an endpoint that is streaming back events at regular intervals; if we stop
and think about it we have laid out the foundation to stream events to a
client, by leveraging manifold.stream/stream</code> abstraction and Aleph's
ability to use it to send back responses to the caller.</p>
I think that implementing a browser client would be even more exciting, so
lets do that! Actually before moving to a browser client I'd like to improve
to server a bit by:</p>

adding a router</li>
that will return an HTML page with all the frontend code</li>
and exposes the SSE handler that we have implemented so far</li>
</ul>
Adding the router</h1>
There are many routers available today, lately I am enjoying using reitit</a>
which I find very intuitive and comes with a comprehensive documentation;
other good alternatives can be compojure</a>, bidi</a> (and many more</a>) or even manual
routing of requests, anyway as you can imagine we will setup out routes with
reitit so we can start by adding it to our deps:</p>
    metosin/reitit {</span>:mvn/version </span>"</span>0.7.0-alpha5</span>"}
</span></code></pre>
(yes it is an alpha version, I am confident that a stable 0.7.x will be out soon)</p>
and setup the router to serve the static page holding the JS code and one
endpoint to handle the SSE events:</p>
    (</span>ns</span> codes.fpsd.sse
</span>      (</span>:require </span>[aleph.http </span>:as</span> http]
</span>                [manifold.stream </span>:as</span> s]
</span>                [reitit.ring </span>:as</span> ring]))
</span>    
</span>    (</span>defn </span>format-event
</span>      "</span>Return a properly formatted event payload</span>"
</span>      [body]
</span>      (</span>str </span>"</span>data: </span>" body "</span>\n\n</span>"))
</span>    
</span>    (</span>defn </span>sse-events </span>[_req]
</span>      {</span>:status 200
</span>       </span>:headers </span>{"</span>content-type</span>" "</span>text/event-stream</span>"}
</span>       </span>:body </span>(</span>let </span>[counter (</span>atom </span>0</span>)]
</span>               (</span>s/periodically
</span>                </span>1000
</span>                #(</span>format-event </span>(</span>str </span>"</span>Sending event #</span>" (</span>swap!</span> counter inc)))))})
</span>    
</span>    (</span>defn </span>create-app
</span>      "</span>Return a ring handler that will route /events to the SSE handler
</span>       and that will servr  static content form project's resource/public directory</span>"
</span>      []
</span>      (</span>ring/ring-handler
</span>       (</span>ring/router
</span>        [["</span>/events</span>" {</span>:get </span>{</span>:handler</span> sse-events
</span>                           </span>:name ::events</span>}}]]
</span>    
</span>        )
</span>       (</span>ring/routes
</span>        (</span>ring/create-resource-handler </span>{</span>:path </span>"</span>/</span>"})
</span>        (</span>ring/create-default-handler</span>))))
</span>    
</span>    
</span>    </span>;; Web server maangement code to make it easy to start and stop a server
</span>    </span>;; after changesto router or handlers
</span>    (</span>def </span>server_ </span>(</span>atom </span>nil</span>))
</span>    
</span>    (</span>defn </span>start-server! </span>[]
</span>      (</span>reset!</span> server_ (</span>http/start-server </span>(</span>create-app</span>)
</span>                                         {</span>:port 8080
</span>                                          </span>:join? false</span>})))
</span>    
</span>    (</span>defn </span>stop-server! </span>[]
</span>      (</span>swap!</span> server_ (</span>fn </span>[s]
</span>                       (</span>when</span> s
</span>                         (</span>.close</span> s)))))
</span>    (</span>comment
</span>    
</span>      (</span>start-server!</span>)
</span>    
</span>      (</span>stop-server!</span>)
</span>      </span>,</span>)
</span></code></pre>
(Please change the namespace to reflect your project setup)</p>
To recap:</p>

we have an helper function to format SSE events</li>
we have renamed the generic handler</code> to sse-events</code> but it is the same as before</li>
we have created a ring router to serve GETs to /events</code> and the static content</li>
we have added some helper functions to start and stop the server</li>
</ul>
If we evaluate the buffer and start the server we can verify that the /events</code> endpoint
is working as before:</p>
❯</span> curl localhost:8080/events</span> -vv
</span>
</span>
</span># Trying 127.0.0.1:8080&#x2026;
</span>
</span>
</span># Connected to localhost (127.0.0.1) port 8080 (#0)
</span>
</span>> GET </span>/events</span> HTTP/1.1
</span>> Host: </span>localhost:8080
</span>> User-Agent: </span>curl/7.81.0
</span>> Accept: </span>**/**
</span>> 
</span>
</span>
</span># Mark bundle as not supporting multiuse
</span>
</span>< HTTP/1.1 </span>200</span> OK
</span>< Content-Type: </span>text/event-stream
</span>< Server: </span>Aleph/0.6.3
</span>< Date: </span>Wed,</span> 02 Aug 2023 06:29:16 GMT
</span>< Connection: </span>Keep-Alive
</span>< transfer-encoding: </span>chunked
</span>< 
</span>data:</span> Sending event </span>#1
</span>
</span>data:</span> Sending event </span>#2
</span>
</span>^C
</span></code></pre>
To recap:</p>

we have added a router (docs here</a>) for the /events</code> endpoint</li>
we have added a way to serve static files from the jar resources (docs here</a>)</li>
a way to conveniently start/stop the webapp</li>
</ul>
I have decided to avoid some details and provide a pre-cooked solution because
reitit's docs are quite comprehensive, and to focus on SSE.</p>
Frontend</h1>
In the previous step we have added a way to serve static files, now we can add a
basic web page where to render events as we get them. First of all lets see if we
can correctly serve a simple HTML file; paste the following content in resources/public/index.html</code></p>
    <</span>html</span>>
</span>      <</span>body</span>>
</span>        Your events here!
</span>        <</span>div </span>id</span>="</span>events</span>"></</span>div</span>>
</span>      </</span>body</span>>
</span>    </</span>html</span>>
</span></code></pre>
Now start the server and point your browser to http://localhost:8080</a>, you should
something like this:</p>
</p>
Quite boring still…we can try subscribing to the /events</code> endpoint and render
the upcoming events! By coincidence we already have a div</code> with the id events</code>
that we can use for this purpose ;)</p>
What we need to do is:</p>

instantiate an EventSource</code> object that connects to the events</code> endpoint</li>
subscribe to incoming events</li>
render event data inside the events</code> div</li>
</ul>
For simplicity we will embedd the JS code inside the index.html</code> page:</p>
    <</span>html</span>>
</span>      <</span>body</span>>
</span>        Your events here!
</span>        <</span>div </span>id</span>="</span>events</span>"></</span>div</span>>
</span>      </</span>body</span>>
</span>      <</span>script </span>type</span>="</span>text/javascript</span>">
</span>        </span>let </span>source </span>= new </span>EventSource</span>('</span>/events</span>')
</span>    
</span>        </span>source</span>.</span>onmessage </span>= (e) </span>=> </span>{
</span>          </span>let </span>container </span>= document.</span>getElementById</span>('</span>events</span>')
</span>          </span>container</span>.</span>prepend</span>(`</span>Received event with data: </span>${</span>e</span>.data}`)
</span>        }
</span>      </</span>script</span>>
</span>    </</span>html</span>>
</span></code></pre>
If we load http:localhost:8080</a> again we should see something like the following
screenshot, again nothing super exciting but I hope you can feel the potential
of this approach!</p>
</p>
Somehow I suspect that this is not enough yet, maybe we can spice it up with a
super simplified chat room?</p>
Generating and consuming events</h1>
What do you need to do to implement the simplest chat room ever? Our MVP
should at least provide:</p>

an event bus to broadcast the messages to all connected users</li>
an endpoint to consume those messages and send to the event bus</li>
a form where to write our messages</li>
a client that can consume such events</li>
</ul>
Following the previous bullet points I'll start with the event bus.
Manifold provides a convenient implementation of an even bus
available at manifold.bus/event-bus</code>. This provides a way to subscribe
a consumer to a topic (or SSE endpoint) and send messages to a topic
which will be sent to all connected consumers. For simplicity
we will have one single topic the-chat-room</code> but it would easy to
extend this example to support multiple rooms.</p>
Second thing to do is to add an endpoint that will get the chat message
and will push it to the event bus, lets see some code:</p>
    (</span>ns</span> codes.fpsd.sse
</span>      (</span>:require </span>[aleph.http </span>:as</span> http]
</span>                [manifold.bus </span>:as</span> b]
</span>                [reitit.ring </span>:as</span> ring]
</span>                [ring.middleware.params </span>:as</span> params]))
</span>    
</span>    </span>;; the one and only event bus needed for this app
</span>    </span>;; using defonce to be able to possibly evaluate the
</span>    </span>;; full buffer without breaking existing connections
</span>    (</span>defonce </span>event-bus </span>(</span>b/event-bus</span>))
</span>    
</span>    (</span>defn </span>format-event
</span>      "</span>Return a properly formatted event payload</span>"
</span>      [body]
</span>      (</span>str </span>"</span>data: </span>" body "</span>\n\n</span>"))
</span>    
</span>    (</span>defn </span>sse-events </span>[_req]
</span>      {</span>:status 200
</span>       </span>:headers </span>{"</span>content-type</span>" "</span>text/event-stream</span>"}
</span>       </span>:body </span>(</span>b/subscribe</span> event-bus "</span>the-chat-room</span>")})
</span>    
</span>    (</span>defn </span>send-message! </span>[request]
</span>      (</span>let </span>[message (</span>format-event </span>(</span>-></span> request </span>:params </span>(</span>get </span>"</span>message</span>")))]
</span>        (</span>b/publish!</span> event-bus "</span>the-chat-room</span>" message)
</span>        {</span>:status 204
</span>         </span>:headers </span>{</span>:content-type </span>"</span>text/plain</span>"}
</span>         </span>:body </span>""}))
</span>    
</span>    (</span>defn </span>create-app
</span>      "</span>Return a ring handler that will route /events to the SSE handler
</span>       and that will servr  static content form project's resource/public directory</span>"
</span>      []
</span>      (</span>ring/ring-handler
</span>       (</span>ring/router
</span>        [["</span>/events</span>" {</span>:get </span>{</span>:handler</span> sse-events
</span>                           </span>:name ::events</span>}}]
</span>         ["</span>/send-message</span>" {</span>:post </span>{</span>:name ::send-message
</span>                                  </span>:handler</span> send-message!}}]]
</span>    
</span>        {</span>:data </span>{</span>:middleware </span>[params/wrap-params]}})
</span>    
</span>       (</span>ring/routes
</span>        (</span>ring/create-resource-handler </span>{</span>:path </span>"</span>/</span>"})
</span>        (</span>ring/create-default-handler</span>))
</span>       ))
</span></code></pre>
Here is a breakdown of the changes:</p>

we requiring manifold.bus</code> to have access to b/event-bus</code></li>
we instantiate an event bus</li>
the handler sse-events</code> will subscribe users to the chat topic and will send back events</li>
the handler send-message</code> will take the message</code> from the request body and send it to the event bus</li>
we have added a new route send-message</code></li>
we have added a middleware that will parse the request body to the key :params</code> of the request</code> map</li>
</ul>
Quite straight forward, isn't it?</p>
Few pointers if you want to dig deeper in the APIs show here:</p>

Data driven middlewares</a></li>
Manifold's streams and event bus</a></li>
</ul>
Now we can implement the frontend:</p>
    <</span>html</span>>
</span>      <</span>body</span>>
</span>        <</span>p</span>>
</span>          <</span>form </span>onsubmit</span>="</span>event</span>.</span>preventDefault</span>(); </span>return </span>sendMessage</span>()">
</span>            <</span>input </span>type</span>="</span>text</span>" </span>name</span>="</span>message</span>" </span>id</span>="</span>message</span>" </span>placeholder</span>="</span>Your message here</span>"/>
</span>            <</span>input </span>type</span>="</span>submit</span>" </span>value</span>="</span>Send</span>"/>
</span>          </</span>form</span>>
</span>        </</span>p</span>>
</span>        <</span>p</span>>Past messages</</span>p</span>>
</span>        <</span>div </span>id</span>="</span>events</span>">first load</</span>div</span>>
</span>      </</span>body</span>>
</span>      <</span>script </span>type</span>="</span>text/javascript</span>">
</span>        </span>let </span>sendMessage </span>= () </span>=> </span>{
</span>          </span>let </span>input </span>= document.</span>getElementById</span>('</span>message</span>')
</span>          </span>fetch</span>('</span>/send-message</span>',
</span>            {method: '</span>POST</span>',
</span>             headers: {"</span>Content-Type</span>": '</span>application/x-www-form-urlencoded</span>'},
</span>             body: `</span>message=</span>${</span>input</span>.value}`}).</span>then</span>(() </span>=> </span>{</span>input</span>.value = ''})
</span>          </span>return </span>false
</span>        }
</span>    
</span>        </span>let </span>appendMessage </span>= (text) </span>=> </span>{
</span>          </span>let </span>p </span>= document.</span>createElement</span>('</span>p</span>')
</span>          </span>p</span>.</span>appendChild</span>(document.</span>createTextNode</span>(</span>text</span>))
</span>    
</span>          </span>let </span>container </span>= document.</span>getElementById</span>('</span>events</span>')
</span>          </span>container</span>.</span>prepend</span>(</span>p</span>)
</span>        }
</span>    
</span>        </span>let </span>source </span>= new </span>EventSource</span>('</span>/events</span>')
</span>    
</span>        </span>source</span>.</span>onmessage </span>= (e) </span>=> </span>{
</span>          </span>appendMessage</span>(</span>e</span>.data)
</span>        }
</span>      </</span>script</span>>
</span>    </</span>html</span>>
</span></code></pre>
Yeah the code looks early 2000 era of HTML+JS (if we exclude the use of fetch</code>),
but I think it shows the core concepts nicely:</p>

we now have a form where we can write our message</li>
we prepend messages to the events</code> div as soon as we will receive them</li>
we use fetch</code> API to POST messages to the send-message</code> endpoint (wow a SPA without React!)</li>
</ul>
Again sorry for the JS code quality but it is not my bread and butter and this is
the best I can do.</p>
After restarting/re-evaluating our backend and loading the page we will be able to
chat in realtime, no too bad for less than 100 lines of code!</p>
</p>
Conclusions</h1>
I hope you had fun following this simple recipe on how to add a bit of real
time feel to a basic web app! SSE, while simple from the outside, can be handy
in a lot of situations and adding it to your web application can be quite easy.
I am sure that the same can be achieved with other web servers and
core.async</code>, if you'll try that approach please write about your experience
and share it with us!</p>
The full source code is available in my playground here</a>.</p>
Discuss</h1>

Reddit</a></li>
Clojurians Slack</a></li>
Twitter</a></li>
Linkedin</a></li>
Fediverse</a></li>
</ul>


Migrating to Zola SSG
FPSD — Tue, 25 Jul 2023 00:00:00 +0000
Overview</h1>
Recently I switched from Haunt</a> to Zola</a> as the static site generator for
this blog. Main reason was a bit of frustration with Haunt's documentation
and the CommonMark implementation it was using which was holding me back.</p>
Why the switch</h1>
When starting this blog I wanted something simple and potentially hackable,
in terms of being able to play with the generator like any other Lisp program,
modifying it a runtime and seeing the result of the change right after it happened.</p>
While Haunt is totally capable of doing so the first roadblock (at least for me)
is that it is written in Guile</a> which I don't know at all; given that it is a
Lisp I hoped to get into it quickly, but I've never studied it or played with it
too much, and there is only so much time in a day so I cannot play with all the
fun things that there are around and at the same time hope to get some work done!</p>
So I started looking for alternatives, maybe leaving aside the hackability of the
underlying system in favor of focusing on writing content.</p>
Don't get me wrong, Haunt is very nice and customizable, it is not just a toy and
for example it is used to build the Guix</a> website (and others); it is possible to
look at that site sources to get an idea on how to write a great website!
The problem is that I did'n manage to learn few fundamental things, Guile and
Haunt internals, quickly enough to be confident with this system and be productive
with it. Better luck next time, maybe.</p>
Candidates</h1>
Cryogen</a></h2>
Language: Clojure
Documentation: While complete the feeling is that it does not flow too well</p>
Pros:</p>

You have the engine at your hand and you can hack on it easily while running</li>
Easy to start</li>
</ul>
Cons:</p>

Custom front matter format (EDN)</li>
Takes some time to get into it, not ideal if you just want a quick blog with your style</li>
Documentation could be better</li>
</ul>
Nikola</a>/Pelican</a></h2>
Language: Python
Documentation: Complete</p>
Pros:</p>

Battle tested</li>
Extensible (with lots of extensions)</li>
Lots of themes</li>
</ul>
Cons:</p>

Custom front matter</li>
Too big for my needs</li>
</ul>
Hugo</a></h2>
Language: Go
Documentation: Complete</p>
Pros:</p>

Battle tested</li>
Extensible (with lots of extensions)</li>
Lots of themes (I mean LOTS!)</li>
Comes packaged in a single binary</li>
</ul>
Cons:</p>

I could not find a clear path to get to my first post quickly</li>
Having a lot of features I felt lost in the docs, which caused friction/churn</li>
Too big for my needs, the home page says "The world’s fastest framework for building websites"</li>
</ul>
Zola</a></h2>
Language: Rust
Documentation: Complete</p>
Pros:</p>

Documentation was feeling just right for my needs</li>
The home pages says "Your one-stop static site engine", just what I need</li>
Quick time to first post and custom layout</li>
Lots of themes (and more are coming to it)</li>
Comes packaged in a single binary</li>
</ul>
Cons:</p>

Maybe not as feature rich as others (not a problem for me)</li>
</ul>
Final decision</h1>
Admittedly I did not run a very scientific comparison process, what I did was
to try to run a generator with its defaults, hack it a bit and measure:</p>

time to get to the same layout and a first post</li>
level of frustration while doing so</li>
documentation</li>
…gut feeling</li>
</ul>
What I did not care about:</p>

speed of generation of content</li>
programming language</li>
availability of themes</li>
</ul>
Following the principle of "least resistance path" I went with Zola because
the process to get to that first post was really painless! Some engines were
not properly documented, with other I felt overwhelmed by options or in the
end it did not feel just right.</p>
It took me 10 minutes to get Zola's approach and file structure, another 30
minutes of grunt work to port the front matter of posts from Haunt (I could
have automated it but it was too hot that day and I was not able to think too
much, but it would have been a nice exercise with Babashka!).</p>
While others may be more complete solution to build complex websites, what I
really want is a tiny container for my posts, potentially some other pages
like contact section, bio etc. and Zola was able to deliver in just no time.</p>
Your requirements may be totally different and other engines may work better
for you, all I can suggest is to try (some of) them and see which one sticks
to you!</p>


Clojure Bites - Ring basic auth
FPSD — Thu, 06 Jul 2023 00:00:00 +0000
Overview</h1>
I am on vacation so I have decided to take it easy and work on small
things on Unrefined</a>; one item in the backlog seemed quite approachable:
adding an admin panel to the project, which requires, among other things,
a way to authenticate users to access those restricted sections.</p>
Here is a quick intro to how to add Basic Auth support to a Ring</a> web
application, using ring-basic-authentication</a> middleware, and, as a bonus,
an intro to clojure.core.cache</a> library.</p>
Project setup</h1>
To show how the middleware works we can create a simple web application with
just one endpoint which later we can protect with basic auth. Start a new
project or use your favorite playground and add the following dependencies:</p>

ring/ring-core 1.10.0</li>
ring/ring-jetty-adapter 1.10.0</li>
ring-basic-authentication/ring-basic-authentication 1.2.0</li>
</ul>
Feel free to use your preferred project and dependency management tool, here I
show how to do that with tools-deps, it is easy to translate it to lein</a> or boot</a>:</p>
{</span>:paths </span>["</span>src</span>" "</span>resources</span>"]
</span> </span>:deps </span>{org.clojure/clojure {</span>:mvn/version </span>"</span>1.11.1</span>"}
</span>        ring/ring-core {</span>:mvn/version </span>"</span>1.10.0</span>"}
</span>        ring/ring-jetty-adapter {</span>:mvn/version </span>"</span>1.10.0</span>"}
</span>        ring-basic-authentication/ring-basic-authentication {</span>:mvn/version </span>"</span>1.2.0</span>"}}
</span> </span>:aliases
</span> {</span>:dev/repl </span>{</span>:main-opts </span>["</span>-m</span>" "</span>nrepl.cmdline</span>" "</span>&#x2013;middleware</span>" "</span>[cider.nrepl/cider-middleware]</span>"]
</span>             </span>:extra-paths </span>["</span>dev</span>"]
</span>             </span>:extra-deps </span>{cider/cider-nrepl {</span>:mvn/version </span>"</span>0.31.0</span>"}
</span>                          djblue/portal {</span>:mvn/version </span>"</span>0.35.1</span>"}
</span>                          com.github.jpmonettas/clojure {</span>:mvn/version </span>"</span>1.11.1</span>"}
</span>                          com.github.jpmonettas/flow-storm-dbg {</span>:mvn/version </span>"</span>3.3.315</span>"}}
</span>             </span>:jvm-opts </span>["</span>-Dclojure.storm.instrumentEnable=true</span>"]}
</span>  }}
</span></code></pre>
The extra alias :dev/repl</code> is my usual goto setup for development, it is not required
to run the examples but it can be generally handy; feel free to ignore it if you
have your own way.</p>
Now we can create a new src/app.clj (or whatever ns you prefer) with the following
content:</p>
(</span>ns</span> app
</span>  "</span>A namespace used to show how the basic-authentication middleware works</span>"
</span>  (</span>:require </span>[ring.adapter.jetty </span>:as</span> jetty]))
</span>
</span>(</span>defn </span>handler
</span>  "</span>Our basic ring handler function, it takes a request map (unused) and
</span>   return a response map</span>"
</span>  [_request]
</span>  {</span>:status 200
</span>   </span>:headers </span>{"</span>Content-Type</span>" "</span>text/html</span>"}
</span>   </span>:body </span>"</span>Hello World</span>"})
</span>
</span>;; We want to hold the server instance in order to close it once done.
</span>;; There are better ways to do it, using life cycle/state management libraries
</span>;; like mount, integrant, component just to name few, but lets keep it simple
</span>(</span>def </span>server-instance </span>(</span>atom </span>nil</span>))
</span>
</span>(</span>defn </span>start-server
</span>  "</span>Starts a web server holding its instance in the server-instance atom</span>"
</span>  []
</span>  (</span>reset!</span> server-instance
</span>          (</span>jetty/run-jetty</span> handler {</span>:port 3000
</span>                                    </span>:join? false</span>})))
</span>
</span>(</span>defn </span>stop-server
</span>  "</span>If there is a server running, stop it and reset the server-instance atom</span>"
</span>  []
</span>  (</span>swap!</span> server-instance
</span>         (</span>fn </span>[inst]
</span>           (</span>when</span> inst
</span>             (</span>.stop</span> inst)))))
</span>
</span>(</span>comment
</span>  (</span>start-server</span>)
</span>  (</span>stop-server</span>)
</span>  </span>,</span>)
</span></code></pre>
If we start a REPL and evaluate the buffer (file) we can start the server by
evaluating the fist form in the comment</code> form, and pointing a browser to
http://localhost:3000</a> we should be able to see something like this:</p>
</p>
Protecting the route</h1>
The library ring-basic-authentication</code> offers the ring middleware
wrap-basic-authentication</code> that will call a function accepting a username
and a password and returns a truty value on success (authenticated)
or a falsy value otherwise. On success the next middleware (or handler) will be
called, otherwise it will return a 401 response.</p>
The first thing we can do is to write a function that will authenticate (or not) a
request based on provided username and password, here is the simplest implementation
we can write:</p>
(</span>defn </span>authenticated?
</span>  [username password]
</span>  (</span>and </span>(</span>= </span>"</span>secret-username</span>" username)
</span>       (</span>= </span>"</span>secret-password</span>" password)))
</span></code></pre>
Now it is time setup ring to use the middleware to authenticate requests. Lets
re-write the start-server</code> function to do that:</p>
(</span>defn </span>gen-app
</span>  []
</span>  (</span>wrap-basic-authentication</span> handler authenticated?))
</span>
</span>(</span>defn </span>start-server
</span>  "</span>Starts a web server holding its instance in the server-instance atom</span>"
</span>  []
</span>  (</span>reset!</span> server-instance
</span>          (</span>jetty/run-jetty </span>(</span>gen-app</span>) {</span>:port 3000
</span>                                      </span>:join? false</span>})))
</span></code></pre>
The utility function gen-app</code> returns a new handler using the wrap-basic-authentication</code>
middleware to wrap our request handler; start server calls the gen-app</code> function
when starting the server, to get the application handler function.</p>
We can see it in action in the following screenshot</p>
</p>
Now for each request this is the flow:</p>

ring calls the app handler</li>
the basic auth middleware is called and</li>
if successful it will return the result of calling the wrapped handler</li>
if not it will return a 401 without even calling the wrapper handler</li>
</ul>
In ring world this is the usual flow; in general we can expect to have chain of
middlewares before the call to the request can even start, common middlewares can
be used for:</p>

request routing: given a path call a specific handler</li>
request parameter parsing: parsed parameters, coming from query string, body etc are injected for later use</li>
response transformation: for example to serialize to JSON, XML or other formats</li>
session, authentication and authorization handling</li>
request logging and metrics</li>
</ul>
There are quite a lot of already existing and useful middlewares that can be
used off the shelf.</p>
Even if this approach can be good enough for some small and private services it
can become unusable or hard to maintain pretty easily if the service grows or must
be exposed to the public internet; few issues that can come to mind:</p>

Hardcoded credentials: even if the project sources can be private it is easy to leak them, better to load them from another source</li>
If we want to change credentials a new build is required; not the end of the world but not really convenient</li>
There is no way to have different handlers and URL with or without the basic auth</li>
</ul>
Lets make it flexible</h1>
First thing we can do to improve this example is to get rid of those ugly hardcoded
credentials.</p>
One way could be to load the username and password from environment variables; as one
extra step would be to on the safe side we can avoid having defaults so that if
we forget to set the environment variables no one can access the restricted
resources. Here is one possible way to authenticated?</code> for our new requirements:</p>
(</span>let </span>[secret-username (</span>:basic-auth-username</span> env "")
</span>      secret-password (</span>:basic-auth-password</span> env "")]
</span>  (</span>defn </span>authenticated?
</span>    [username password]
</span>    (</span>and </span>(</span>seq</span> secret-username) </span>;; making sure this two are set to something
</span>         (</span>seq</span> secret-password) </span>;; TIL that (seq "") is equivalent to (not (empty? "")), thanks clj-kondo!
</span>         (</span>=</span> secret-username username)
</span>         (</span>=</span> secret-password password))))
</span></code></pre>
Wait wait wait, where is this env</code> coming from?</p>
Even if is possible to get the value of environment variables using Java
interop via System/getenv</code> I have decided to introduce the library environ</a>
because it offers a Clojure only interface and provides more goodies like
reading environment variables from env files or JVM properties and has a
nicer interface (IMHO).</p>
To be able to use this library all we need to do is
to add it to our deps.edn</code> (or…you know) and refer the env</code> symbol in our
namespace. For more details please refer to the library docs, all we need to
know now is that it behaves like a normal map, at least for reading.</p>
Oh, one more thing, keywords are translated to "UPPER_{SNAKE</sub>_{CASE</sub>" when looking
them up, in our case :basic-auth-username</code> will lookup BASIC_AUTH_USERNAME</code>
environmental variable.</p>}}
Even more flexible</h1>
Now we want to give access to the protected resource to more people and usually
sharing the same credentials is not a good practice; what happens if we want
to invalidate the access for one of the users? We should create new credentials
and share it only with the ones whom should have access to the resource. It is
not really practical, is it? So at this point we may want to have user specific
credentials but handling them with env vars can be too complicated or unpractical.</p>
An option is to store credentials in a file that we can read at the start of the
application, in case someone leaves our organization we can update our credentials
file and restart the app. For convenience we can read the credentials file path
from an environment variable, we already know how to do it right? Right.</p>
(</span>defn </span>get-credentials
</span>  "</span>Return a username -&gt; password map reading it form the specified file, or an
</span>   empty map it reading fails</span>"
</span>  [credentials-path]
</span>  (</span>try
</span>    (</span>with-open </span>[r (</span>io/reader</span> credentials-path)]
</span>      (</span>edn/read </span>(</span>java.io.PushbackReader.</span> r)))
</span>    (</span>catch</span> Throwable  _ {})))
</span>
</span>(</span>defn </span>authenticated?
</span>  [username password]
</span>  (</span>let </span>[credentials (</span>get-credentials </span>(</span>:basic-auth-credentials</span> env "</span>credentials.edn</span>"))]
</span>    (</span>= </span>(</span>get</span> credentials username) password)))
</span></code></pre>
Now authenticated?</code> will read the credentials from an edn file whose path is taken
from the BASIC_AUTH_CREDENTIALS</code> environment variable (defaulting to some well
known path). If the credentials will ever change we must restart the application
to see the new values, not the end of the world but we can do a bit better.</p>
The idea is to cache the credentials with a TTL, so when it expires we can read
the file again with possibly new values. Implementing a cache with TTL is a nice
exercise but we can leverage the clojure.core.cache</a> library that implements
this functionality for us. I am not going to write about this library in detail,
the documentation does a great job anyway, so I'll focus on the bare minimum
to get things done.</p>
Assuming that we have added the library to our dependencies and required it in
our namespace, we can change autheticated?</code> to make use of it:</p>
;; create a TTL cache with an empty map and a ttl of 60 seconds
</span>(</span>def </span>credentials-cache </span>(</span>cache/ttl-cache-factory </span>{} </span>:ttl 60000</span>))
</span>
</span>(</span>defn </span>authenticated?
</span>  [username password]
</span>  (</span>let </span>[credentials (</span>cache/lookup-or-miss
</span>                      credentials-cache
</span>                      </span>:credentials </span>;; we try to lookup the key :credentials in the cache
</span>                      (</span>fn </span>[_]      </span>;; if the key is missing or expired we load the credentials file
</span>                        (</span>get-credentials </span>(</span>:basic-auth-credentials</span> env "</span>credentials.edn</span>"))))]
</span>    (</span>= </span>(</span>get</span> credentials username) password)))
</span></code></pre>
This is a sorts of an hack of the TTL cache; usually you want to cache with TTL different keys,
and load the value of those keys (from DB, external APIs, whatever) when the TTL expires. In our
case we just want to load the whole file so we are using the key :credentials</code> to hold its
content. The clojure.core.cache implements quite a lot of caching strategies, I encourage everyone
to give it a try!</p>
Conclusions</h1>
If dealing with a simple application, using basic auth can be good enough
and we have explored different ways to achieve that; clearly, as the application
grows, other authentication mechanisms like a full suited identity/role management
solution may be a better fit. Currently I am exploring Keycloak</a>, hoping to come
up with a library for Clojure.</p>
Sources for this little project can be found in my shiny new playground repo</a>,
specifically in the ring_basic_auth.clj</a> file</p>
Alternatives</h1>
Every web server provides a way to protect one (or more) endpoint at the
server configuration level, sometimes it can be even convenient to
leverage that functionality; the only downside I see in this approach is
that it couples the application to the web server serving it.
If the endpoint structure is sufficiently complex it is easy to forget
something in case of a migration to a different web server.</p>
Another alternative is to use the same authentication/authorization mechanism
used for normal users and play with roles and permissions to enable/disable
access to specific resources. In the longer term this is the approach I
would take given that it reduces special cases and makes role management
more consistent.</p>
Discuss</h1>

Reddit</a></li>
Dev.to</a></li>
Twitter</a></li>
Linkedin</a></li>
</ul>


Skateboarding, keep track of your progress
FPSD — Thu, 29 Jun 2023 00:00:00 +0000
Overview</h1>
Keep track of your skateboarding progress, set your goals
and create a plan to achieve them at your own pace.</p>
Possible names</h1>

skoach</li>
coach360</li>
pops</li>
scoop</li>
</ul>
Application platform</h1>

website, mostly for offline activity like loading media, set plans</li>
mobile app, for real time updates</li>
</ul>
Features</h1>

record where you are, tricks you own trick you want to get</li>
set monthly/weekly/daily plans and get stats</li>
gamification, achievements, challenges</li>
suggest plans and workout activities</li>
</ul>
Copy for landing</h1>
Random ideas</h2>
From skateboarders to skateboarders.</p>
Land that trick that is always escaping you, keep track of your
progress, learn from your mistakes.</p>
Skateboarding is freedom, and that great feeling when you land a
trick.</p>
Keep yourself accountable, share your progress with your crew, set
personal and group challenges.</p>
If it is not recorded it didn't happen, record yourself and share
the videos on social media platforms.</p>
Achieve your goals and unlock new challenges.</p>
Remember to have fun!</p>
Business model</h1>

Freemium</li>
Pro with subscription or LTD</li>
</ul>
Longer term vision</h1>

For amateurs going pro</li>
For teams, either brand or national</li>
Live coaching, talk to a pro, show videos and get help</li>
Workout coach</li>
Tools for content creation</li>
Public challenges with sponsors</li>
</ul>


Just release Unrefined's commit 300, an update!
FPSD — Sun, 25 Jun 2023 00:00:00 +0000
Just deployed commit 300!</h1>
I was working on Unrefined, tested some changes, fixed a bug, cooked a release
and suddenly I've realized that it was commit 300! This requires an update post
for sure ;)</p>
Bugs</h2>
We use Unrefined</a> at work and from time to time we experience some weird
behaviors, usually nothing too serious the blocks us but at least one bug
was ruining a refinement session.</p>
We were refining a ticket together with the product manager and we found that
it had to rewritten to make it possible to estimate it. The ticket was broken
down to smaller one and when we came back to the original one, as usual we
started the process to estimate it again and Unrefined start throwing exceptions!
The problem was that a refinement + ticket id are defined as unique and trying
to start a new session for that ticket raise an exception from the DB layer.
We worked around this issue but I was not too happy, so I've created an issue
which I've closed today.</p>
Restructuring code</h2>
The namespaces that make up the application were all over the place and did
not use the pattern "tld.your-domain.app-name" so I've restructured everything
and now all the namespaces are living inside "one.unrefined" namespace; it is
not the most important change in the world but given that I've started working
on the frontend side with the correct namespace model I wanted to be consistent.</p>
ClojureScript</h2>
I've started working on the cheatsheet customization code for the frontend, it
started as a separate project to experiment with it, choosing the framework
(the good old rum</a> or helix</a>? went with the familiar one, rum) and the build system
trying out shadow-cljs over the more familiar figwheel-main. After some try and
error I've decided to stick with shadow-cljs because it will be easier to pull
in Javascript dependencies. I still like figwheel-main especially if used with
ClojureScript only solutions (for example dumdom</a>).</p>
The custom cheatsheet is still in WIP state and I hope to release it as
soon as possible, it should be a game changer for Unrefined, preparing the road
for user accounts.</p>
WIP</h2>
Some parts of Unrefined are poorly or not tested at all, the core logic is but
I'd like to change this in the near future. It will be almost mandatory as soon
as more users and organizations will use this tool.</p>
Also pending, and depending on user auth, is some sort of SSO or OAuth to leverage
existing platforms like Jira, Asana and friends.</p>
Closing</h1>
My goal with this post is to give an update on Unrefined and what is coming next,
it could have been a tweet but it is too long :D</p>
See you on commit 400! But most probably even earlier ;)</p>


Clojure digressions - Setting up a playground
FPSD — Fri, 23 Jun 2023 00:00:00 +0000
Disclaimer</h1>
Why a new series named Clojure Digressions?</p>
Clojure Bites is more suited for short introductions to Clojure
related libraries and tools, instead with digressions I'd like to
talk about more general topics, like development environment setup,
good/bad habits and personal development as a Clojure developer.</p>
Like a conversation ice breaker, I start by talking about my personal
take on some specific argument, looking forward to get input from
other, possibly more experienced, developers on some specific topic.</p>
Overview</h1>
A playground is a safe place aimed at experimentation, without any
legacy background limiting your fantasy and willingness to explore
solutions outside your usual comfort zone i.e. all the code you have
already written for your company/product.</p>
It is no secret that as professional developers we are expected to reuse
as much as possible of what we have at hand in our code base, being careful
when adding new dependencies, limit experimentation to when it is really
really needed to avoid introducing unforeseen costs but at the same time we
love to try out new things! What if we had a safe place to experiment,
just taking out a small bit of our business domain and test it against
a different approach?</p>
Starting from scratch</h2>
You want to quickly sketch out something, no REPL is running, the first
thing you can do is to fire up an interactive REPL by running $ clojure</code>
or, if you want readline support, $ clj</code>; now you have a prompt where you
con evaluate forms using the rich clojure.core library. Nothing too advanced,
the same is available with the default Python, Ruby and many other prompts.
But it is still something.</p>
Lets spice up our REPL by replacing the standard readline lib with
rebel</a> that will give us autocompletion,
hints and much more. To try it out quickly:</p>
$</span> clojure</span> -Sdeps </span>"</span>{:deps {com.bhauman/rebel-readline {:mvn/version </span>\"</span>0.1.4</span>\"</span>}}}</span>"</span> -m</span> rebel-readline.main
</span></code></pre>
The `-Sdeps` is used to pull in more dependencies, the `-m` parameter is used to
specify an entry point to be called; this will initialize a REPL session using
rebel readline interface. Learn more here</a>.</p>
If you'll like it you can create an alias in your `~/.clojure/deps.edn` file in
order to be able to start a rebel REPL without being forced to remember all the
required parameters, here is an example:</p>
{
</span>  </span>:aliases </span>{</span>:rebel </span>{</span>:extra-deps </span>{com.bhauman/rebel-readline {</span>:mvn/version </span>"</span>0.1.4</span>"}}
</span>                    </span>:main-opts  </span>["</span>-m</span>" "</span>rebel-readline.main</span>"]}}
</span>}
</span></code></pre>
And invoke it simply with:</p>
$</span> clojure</span> -M</span>:rebel
</span></code></pre>
The asciinema</a> of the official docs is worth one
billion words ;)</p>
Beyond throw away sessions</h2>
Working directly in the REPL prompt is not really ergonomic in some cases and you may
prefer the comfort of your favorite IDE or editor to hack on some code; the first
step is to start a REPL to which you can attach to. As with rebel readline it not
required to to have an alias in your $HOME/.clojure/deps.edn or project's deps.edn:</p>
$</span> clojure</span> -Sdeps </span>"</span>{:deps {nrepl/nrepl {:mvn/version </span>\"</span>1.0.0</span>\"</span>}}}</span>"</span> -m</span> nrepl.cmdline
</span>
</span>nREPL</span> server started on port 39435 on host localhost - nrepl://localhost:39435
</span></code></pre>
Now you can connect your IDE to the REPL and start hacking! Please notice that the
port may change at each invocation. Deciding what to work on is left as an exercise
to the reader :)</p>
Like we did for rebel readline, we can add an alias to our $HOME/.clojure/deps.edn
file to make it easier in future to start a REPL we can attach to:</p>
{
</span>  </span>:aliases </span>{</span>:rebel </span>{</span>:extra-deps </span>{com.bhauman/rebel-readline {</span>:mvn/version </span>"</span>0.1.4</span>"}}
</span>                    </span>:main-opts  </span>["</span>-m</span>" "</span>rebel-readline.main</span>"]}
</span>            </span>:nrep  </span>{</span>:extra-deps </span>{nrepl/nrepl {</span>:mvn/version </span>"</span>1.0.0</span>"}}
</span>                    </span>:main-opts  </span>["</span>-m</span>" "</span>nrepl.cmdline</span>"]}}
</span>}
</span></code></pre>
Starting it with:</p>
$</span> clojure</span> -M</span>:nrepl
</span></code></pre>
At this point I encourage you to test this setup with your own IDE to get a real feel
of it.</p>
Some more spices</h2>
Being able to run a REPL on demand is already something useful but we can get more
out of it, for example being able to pull in new dependencies on demand in the
currently running process using the upcoming add-lib</code> in the repl.deps</code> namespace,
see here</a> for more details.</p>
Add a new alias to our clojure deps file:</p>
{
</span>  </span>:aliases </span>{</span>:rebel </span>{</span>:extra-deps </span>{com.bhauman/rebel-readline {</span>:mvn/version </span>"</span>0.1.4</span>"}}
</span>                    </span>:main-opts  </span>["</span>-m</span>" "</span>rebel-readline.main</span>"]}
</span>            </span>:add-libs </span>{</span>:extra-deps </span>{clojure.org/clojure {</span>:mvn/version </span>"</span>1.12.0-alpha3</span>"}}}
</span>            </span>:nrep  </span>{</span>:extra-deps </span>{nrepl/nrepl {</span>:mvn/version </span>"</span>1.0.0</span>"}}
</span>                    </span>:main-opts  </span>["</span>-m</span>" "</span>nrepl.cmdline</span>"]}}
</span>}
</span></code></pre>
Now we can add this capability to either rebel or nrepl alias, for example, running:</p>
$</span> clojure</span> -M</span>:add-libs:rebel
</span></code></pre>
will start a new REPL pulling in both the new Clojure version with support to dynamically
load libraries and a rebel readline prompt; the same would apply if replacing rebel with
nrepl, i.e. we would have a nrepl session with the add-libs support.</p>
It is useful to be able to combine more functionalities coming from different aliases when
starting a new REPL process. Again I encourage everyone to play with it and come with a setup
the covers your requirements.</p>
Other useful tools worth considering for your personal toolbox:</p>

Portal</a>: A clojure tool to navigate through your data.</li>
FlowStorm</a>: A tracing debugger for Clojure and ClojureScript.</li>
ClojureStorm</a>: Companion FlowStorm Clojure implementation that automatically instruments your code to be used with FlowStorm.</li>
deps-try</a>: an add on for rebel-readline to try out new libraries if you don't like the upcoming add-libs.</li>
</ul>
It can be seen as a safe place to play with unknown possible solutions,
in an isolated environment, collect useful information and then decide
if we can introduce them to our daily stack.</p>
How can it be useful?</p>
Which tools should I include in mine?</p>
Is there something obvious that I am missing?</p>
Are we playground yet?</h2>
We have learned how to setup an interactive or a nREPL to connect to from an
IDE and we have added some useful tools to improve our developer experience
but where do we put the results of our experiments? I usually start a new
project each time I want try out something and honestly it is a straight
forward process especially if using tools like clj-new</a>, taking from the project
README:</p>
# one-off to install clj-new as a tool:
</span>$</span> clojure</span> -Ttools</span> install com.github.seancorfield/clj-new '</span>{:git/tag "v1.2.399"}</span>' :as clj-new
</span>
</span># commands to create new projects:
</span>
</span># create a new app:
</span>$</span> clojure</span> -Tclj-new</span> app :name myname/myapp
</span># create a new library:
</span>$</span> clojure</span> -Tclj-new</span> lib :name myname/mylib
</span># create a new template:
</span>$</span> clojure</span> -Tclj-new</span> template :name myname/mytemplate
</span># create a new project from a public template:
</span>$</span> clojure</span> -Tclj-new</span> create :template electron-app :name myname/myelectron-app#+END_SRC
</span></code></pre>
Recently maybe I have created too many throw away projects and creating a full
fledged project structure each time feels like a waste of time; ideally I'd like
to open a new file, connect to a fully capable REPL (or start a new one) and
start hacking. When finished commit the new file or changes to existing ones and
move one.</p>
One advantage I see with this approach is that I can keep all these small projects
in one place, accumulating knowledge that can be easily reused later. An added
benefit is that it is easy to add another powerful tool like Clerk</a> to generate
playbooks to interact with and document the code; two months from now you may
want to get back to your experiment and easily get back to it and its conclusions
without spending too much time.</p>
Conclusion</h1>
We started from a simple REPL, added some tasty juice to it and ended up creating
a safe place to play and experiment. I want to point out that this is just my personal
take and not a default in the Clojure community (or if that is, it is just a coincidence).</p>
I am curious to learn how others are addressing the same problem, if it is a problem anyway.</p>
Discuss</h1>

Reddit</a></li>
Twitter</a></li>
Dev.to</a></li>
Fediverse</a></li>
Linkedin</a></li>
</ul>


Opening wave 2022-06-16
Unknown — Fri, 16 Jun 2023 00:00:00 +0000
Starting Wave 2023-06-16</h1>
After a two days break I am ready to open the new wave, that will last
one month like the previous one, I feel comfortable with this pace!</p>
What's in the plan? It be broken down to the following items.</p>
Before diving into the wave's goals, the usual link to the MR</a> 
that will collect all the work being done and shared later in this site.</p>
Unrefined</h2>
Given the seminal work of the previous wave the plan is to finish the
customization of cheat sheets. The initial implementation of the UI is
quite rough but I can focus on improving it later if there is enough
interest, now it is time to embed it into the app. The extension may
come next but realistically in the next wave.</p>
Another key addition to Unrefined</a>, pending for
too long, is the landing page and some content explaining what problems
it is trying to solve and how, namely estimation bias and why the estimation
cheat sheet is a core function of the flow.</p>
Authentication has been mentioned before and delayed for too long, it is
now time to add it, probably using an external identity management platform
like Clerk</a>.</p>
fpsd.codes</h2>
The plan is to continue publishing at least two Clojure Bite articles and
updates regarding product development; posts about Unrefined will be
replicated in the upcoming application blog.</p>
If possible I'd like to also include updates on the second organization
the should be onboarded to Unrefined, if it will even happen ;).</p>
Redesigning the general layout, like grouping posts be month, adding
contact pages etc will be postponed to the next wave.</p>
Live streaming</h2>
Still unsure how to drive more value while streaming, I guess I'll
keep trying to learn how to make something out of it.</p>
Closing</h2>
This post sets the start of the new wave, the feeling is that the plan
is more clear compared to the previous one, I hope that the closing post
will confirm that it was a good plan, or at least that I've learned how
to improve for the next time, see you around the net!</p>
Discuss</h1>

Twitter</a></li>
Linkedin</a></li>
IndieHackers</a></li>
</ul>


Closing wave 2022-05-17
Unknown — Mon, 12 Jun 2023 00:00:00 +0000
Overview</h1>
Every wave is a great source of learning material, built on top of the
previous experience and the expectations I set for myself. So what
have I learned this time?</p>
To summarize:</p>

reaching out to the builders/solopreneurs community is a force multiplier</li>
time management is still an issue but on its way to be addressed</li>
same for time allocation, should I spend more time writing or building?</li>
</ul>
In this post I'll try to put down an honest analysis of the past wave, and
set the ground for the next one.</p>
Some metrics</h2>
fpsd.codes</h3>
First of all the raw numbers, the blog have reached the all time high of 1K
unique visitors! It does not look like a lot, and it isn't, but I have to
consider that the main traffic comes from Clojure Bites posts, and it is a very
small niche. I enjoy writing those posts, they seem useful in general and not
just related to my effort to create and advertise my products…but they help in
that sense too.</p>
Main traffic sources are:</p>

Reddit</li>
Twitter</li>
Linkedin</li>
Dev.to</li>
Clojure Deref</li>
</ul>
There are other sources but those are both irrelevant and surprisingly I
don't market my posts there! Weird but interesting.</p>
I can confirm that some channels perform better than others depending on
the content of the posts; more technical posts perform great in Reddit
because it is easy to directly reach the interested community. Twitter
performs well too, probably because some aggregators like PlanetClojure</a>
will re-post to interested communities. A new entry in this wave is
Clojure Deref</a>, a weekly update about all things Clojure; it collects,
videos, podcasts, blogs and libraries updates, highly suggested!
To try in future, again for Clojure related posts, is the channel
#news-and-articles in Clojurians Slack</a>.</p>
One trick that I have learned is to publish a Clojure related post on Wednesday
or Thursday in order to be featured in the next edition of Clojure Deref.
Friday works terribly and I haven't tried earlier in the week but I am
usually ready to publish on Wednesdays so, yeah, there is no incentive in
trying other schedules.</p>
Linkedin and Twitter are great places to talk about projects updates and
everything related to the "Build in public" community. I can highly
recommend getting in touch with the people posting with the tag #buildinpublic
on Twitter, and get invited to the "Build, Show and Tell" meetup, the
website</a> is not finished yet, but a great design is coming soon!</p>
Unrefined</h3>
Unrefined user growth is still at 1 user, 0 paying customers. The reason being
that it needs one necessary change, pointed out by many, more on that later.</p>
Engaging with the community</h2>
Just to repeat myself, engage with your community! Look for people living the
problem you want to solve and get in touch with them; if you are building something
it also helps follow the #buildinpublic community on Twitter, amazing people are
gathering to help each other with their own projects in an honest and transparent way.
The community is growing and this is the best time to get in touch with them (us? ;) ).</p>
Last week it was my turn to present the project I am working on (Unrefined</a>) and I've
got the best feedback ever! The presentation and discussion was extremely smooth and
friendly and, believe me, even if you are shy you will feel like you are talking to
old time and supportive friends! Another underrated factor is that you can also
provide great feedback to other people, learn from them and grow all together, so
give yourself a favor and reach out to this amazing community!</p>
Coming back to my personal effort to reach out to communities that could be interested
in using Unrefined, I can say that it has been totally not existent and clearly this
must be fixed in the upcoming months. Note taken…</p>
Time management</h2>
During the past wave I have experienced a bit of stress, trying to do too much at the
same time, especially considering the limited time budget. What has suffered the most
was Unrefined, with almost no development or marketing dedicated to it. During the
current wave, which this post is closing, I have taken a more relaxed approach and I
think it worked; probably I am not the most productive guy on the planet but at least
I've got:</p>

First, fundamental change to Unrefined to make it possible to select a different cheat sheet</li>
One Clojure Bites post every two weeks; I am aiming at one per week but I have to build a queue</li>
Engage with the community of builders which gave me a boost in motivation and productivity</li>
Tried to stream two times; this is a time and energy consuming activity and something is telling me to try it more</li>
</ul>
My plans for the next wave is to stick to this schedule and possibly try to get some
more but it is like a stretch goal. I don't want to feel stressed for no reason and
brake a functioning routine.</p>
Projects</h2>
Unrefined</h3>
This wave have seen the advent of the most requested feature, being able to select
different cheat sheets that may be more relevant to companies routines and processes.</p>
I am currently working on an interface to enable the customization of the cheat sheet,
the second most request feature. After that I want to enable signing in, a foundational
work to give more value to prospect paying customers. This would enable providing stats
on past refinements but the value is yet to be validated.</p>
fpsd.codes</h3>
The goal with this website is to stick to at least one Clojure Bites every other week,
and possibly add more content; I'd like to give more updates on Unrefined or other
projects (especially in the style of small bets).</p>
I have a backlog of changes regarding the layout structure, post grouping, enabling comments
and so on but lets see. I have some free days at the end of the month, I hope to make use
of them!</p>
One idea in the back burner is a new series "Clojure Meals" covering bigger topics or deep
dives. Maybe Once a month? Every other month?</p>
Streaming</h3>
This is a new thing for me, I am learning everything from scratch and it is time consuming,
for each minute spent streaming I think it costs 30 minutes to 1 hour of preparation. The
value I can see is that it forces me to build in public and get in the flow, I'd like to have
some form of interaction but there is no audience yet LOL. I'll try it some more just to see.</p>
Closing words</h2>
I am still learning my way out there in the wild word of internet, I am having fun and
learning. Maybe all this effort will not be useful in the near future but I bet on the
compounding effect of all activities.</p>
This post closes the wave 2023-05-17, see you on the next wave!</p>


Clojure bites - Rendering HTML
FPSD — Thu, 08 Jun 2023 00:00:00 +0000
Overview</h1>
There are many ways to produce HTML in Clojure, from template engines
to data oriented DSLs and React wrappers. Each one has it own use cases,
benefits and downsides so there is no one solution that fits all problems.</p>
In this post we will have a look at selmer</a>, a library that offers a well known approach for HTML templating, by the prolific 
Dmitri Sotnikov</a> (AKA Yogthos).</p>
Selmer</h2>
Since the beginning of time, template engines have ruled the world of web
development, in a form or another. That is how the majestic PHP has come to
life, and it is still out there and kicking.</p>
Other notable mentions in the same space are Jinja for Python, Erc for Ruby,
Mustache/Handlebars for JS, and probably many, many more for all of the
available languages anyone can think of.</p>
In Clojure world selmer</a> is a very nice library that will look familiar to Jinja
users, something to consider if you are coming from that kind of experience.</p>
Pros of this kind of approach:</p>

knowledge is easily transferable between languages</li>
familiarity of templating language</li>
easy interop with web designers, they provide you the HTML and you enrich it with dynamic content</li>
easy to use</li>
</ul>
What follows in an example of how a template may look like, assuming that we
want to render the index of a blog, with a list of posts and a preview of their
content; do not worry, all the building blocks will be explained later.</p>
  <</span>h1</span>>{{blog.title}}</</span>h1</span>>
</span>  <</span>p</span>>Latest posts</</span>p</span>>
</span>  {% for post in blog.posts %}
</span>  <</span>h2</span>>{{post.title}}</</span>h2</span>>
</span>  <</span>p</span>>{{post.sections.0.body|abbrev</</span>p</span>>
</span>  <</span>a </span>href</span>="</span>{{post.url}}</span>">Full article</</span>a</span>>
</span>  {% endfor %}
</span></code></pre>
In this example are visible to main (and only) building blocks of the template
engine offered by selmer:</p>

{{ placeholders }}</li>
{{ placeholder|filters }}</li>
{% tags %}</li>
</ul>
Placeholders</h3>
Placeholders are identified by double curly braces, for example {{blog.tile}},
and essentially take the content of a variable and render it to the output,
taking from the blog example:</p>
  <</span>h1</span>>{{blog.title}}</</span>h1</span>>
</span></code></pre>
Assuming that title's value is "My awesome heading", the generated output will
be:</p>
  <</span>h1</span>>My awesome heading</</span>h1</span>>
</span></code></pre>
Placeholders can also access nested data structures like maps and sequences
using the dot notation; building on top of the previous example assume that
we have a map representing a blog post with title and sections and we want
to render a preview, showing just title and the first section, example map</p>
  {</span>:title </span>"</span>A blog post</span>"
</span>    </span>:sections </span>[{</span>:title </span>"</span>Section 1</span>"
</span>                </span>:body </span>"</span>Some very long text</span>"}
</span>              {</span>:title </span>"</span>Section 2</span>"
</span>                </span>:body </span>"</span>Some other long text</span>"}]}
</span></code></pre>
And the preview template, assuming that we want to show the preview of the
first section of a post.</p>
  <</span>h1</span>>{{post.title}}</</span>h1</span>>
</span>  <</span>h2</span>>{{post.sections.0.title}}</</span>h2</span>>
</span>  <</span>p</span>>{{post.sections.0.body}}</</span>p</span>>
</span></code></pre>
When rendered, the output will look like this</p>
  <</span>h1</span>>A blog post</</span>h1</span>>
</span>  <</span>h2</span>>Section 1</</span>h2</span>>
</span>  <</span>p</span>>Some very long text</</span>p</span>>
</span></code></pre>
Filters</h3>
Filters can be piped to placeholders to alter their values before being sent
to the output, the syntax is {{placeholder|filter1|filter2:extra_param|…filterN}}.</p>
These are essentially functions that takes as the first argument the result of the
previous step, plus other optional parameters, and return a value to either be
rendered or piped to the next filter. There are a lot of builtin filters and it is
possible to add custom ones.</p>
Here</a> is a list of the builtin filters.</p>
Tags</h3>
Tags provide a way to introduce logic inside templates, offering conditionals,
iterations and composition via inheritance and fragment template loading, which
enables template code reuse.</p>
Commonly used tags are:</p>

for (seen it in the first example)</li>
if: conditionally render some part of the template</li>
include: takes another template and renders it in the current one</li>
extend: enables template inheritance</li>
block: commonly used with inheritance, make it possible to inject code in a parent template</li>
</ul>
There are many more and like filters, tags can be extended to add more functionalities
to your templates.</p>
Please refer to the official docs</a> for more details.</p>
As a side note</a>, selmer can be used out side
the HTML rendering context, as a generic template engine even if I
focus on this aspect in this post.</p>
Other options</h2>
Of course there are other options available but I am not going to cover them in
this post. Worth mentioning:</p>

Hiccup</a>, Sablono</a>: data oriented DSL commonly used in frameworks like rum or reagent</li>
Helix</a>: thin wrapper around modern React, uses react/dom to render components</li>
Clostache</a>: mustache for Clojure; looks a quite unmaintained but maybe you can resurrect it!</li>
</ul>
Let me know if I have missed something or if you want a deep dive of another library,
I am constantly looking for new topics to cover in this humble website.</p>
Discuss</h2>

Reddit</a></li>
Dev.to</a></li>
Twitter</a></li>
Linkedin</a></li>
</ul>


Clojure bites - Structured logging with mulog
FPSD — Wed, 24 May 2023 00:00:00 +0000
Overview</h1>
Logging is a fundamental tool to monitor and debug a running system.
Even if we can use logs to gather metrics about our system, these are
often written with the assumption that humans are going to consume them,
which make it hard to extract meaning information from log messages.</p>
Log messages are rarely consistent or even meaningful if read few days
after writing them during your emergency debug session. Writing good log
messages is hard! Almost as hard as naming things and cache invalidation.</p>
Another overlooked problem is that often logs are are context less, what
can we infer by a message like "Failed to process user payment" if we don't
know what triggered the payment process, the affected user or product?</p>
Structured logging</h2>
Structured logging aims to provide query-able, consistent, information rich
logs that can be used for:</p>

Business intelligence: derive business relevant data from logged events</li>
Monitoring: understand the current state of a system</li>
Debugging: understand the context in which an error has been reported</li>
</ul>
mulog</h2>
mulog</a> is a Clojure library, developed by Bruno Bonacci, that can be used
to replace your text based logs with content rich events, enabling a large
set of analytics for the data produced by your system. I highly suggest to
read the motivation</a> behind this library to understand the author's vision
but, for the impatient, it can be summarized as following:</p>

Text based logs are hard to write and hard to parse</li>
Logs are generally written with the assumption that humans will consume that, making them less useful for analytics</li>
Text logs often have no context</li>
Logs with extra data are serialized as text and de-serialized when ingesting them, a clear waste of resources</li>
</ul>
Setup</h2>
Without further ado, lets jump to some examples!</p>
We can either create a new project adding mulog to the dependencies
or start a REPL with support to clojure.repl.deps/add-lib and
download the lib dynamically to our session [older post with add-lib].</p>
New project</p>
  </span>$</span> clj</span> -X</span>:new-app :name mulog-example.main
</span>  </span>$</span> cd mulog-main
</span></code></pre>
The :new-app alias is defined in my $HOME/.clojure/deps.edn, using the
handy clj-new</a> tool by Sean Corfield; the project is well documented and
the README is a great place to get you up and running, in case you are
interested to try it out. It is also possible to start from scratch with
a basic deps.edn file like the following:</p>
  {</span>:paths </span>["</span>src</span>"]
</span>    </span>:deps </span>{org.clojure/clojure {</span>:mvn/version </span>"</span>1.11.1</span>"}
</span>           com.brunobonacci/mulog {</span>:mvn/version </span>"</span>0.9.0</span>"}}}
</span></code></pre>
Assuming a fully functional project/REPL/whatever setup with mulog
available to you it is time to write our first log:</p>
  (</span>require </span>'[com.brunobonacci.mulog </span>:as</span> u])
</span>  (</span>u/log </span>::my-first-event :message </span>"</span>a log, wow!</span>")
</span></code></pre>
I want to point out few things:</p>

the first parameter is the event name, it is a good practice to use a fully qualified keyword</li>
the event name is followed by as many key/value pairs as we need to enrich the log event</li>
if we evaluate that code nothing is logged anywhere! what?</li>
</ul>
Publishers</h2>
mulog make it possible to send events to different publishers which will take
care of delivering the events to the right destinations. Out of the box mulog
supports console and file publishers but there are my others available to be
plugged in. It is also possible to write your own publisher if your target tools
is not supported yet. A list of currently supported publishers is available here</a>.</p>
For the sake of this example we can enable the console publisher</p>
  (</span>u/start-publisher! </span>{</span>:type :console</span>}) </span>;;  good enough for development
</span></code></pre>
After evaluating it, we will be able to see the event logged to our REPL.
By default an event will look like the following map:</p>
  {</span>:mulog/trace-id   </span>#mulog/flake "</span>4q6x5wlEnU0WNV5fV0HTvJ32Lz3b4GvV</span>"
</span>    </span>:mulog/timestamp  1684906031447
</span>    </span>:mulog/event-name :your-ns/my-first-event
</span>    </span>:mulog/namespace  </span>"</span>your-ns</span>"
</span>    </span>:message          </span>"</span>a log, wow!</span>"}
</span></code></pre>
Keys in the mulog namespace are automatically added by the library, the
rest is whatever we have provided as extra parameters in the call.</p>
Context</h2>
Enriching our logs with custom data is the first step towards our goal
to improve the observability of our systems. Another key factor is the
consistency of the data collected with logs.</p>
As systems grow in size and complexity it might be hard and tedious
to maintain the same set of keys across all functions and entry points.</p>
Assuming that we are writing a web application we may want to keep a
standard set of key across all event generate by all handlers, for example
user id, ip, endpoint path or name and so on.</p>
mulog events are logged within a context and we have already seen it in
action in our first log, and it was represented by the keys in mulog
namespace.</p>
It is possible to add more data to the logging context using:</p>
   </span>;; (u/set-global-contex! {:key1 val1 &#x2026; keyN valN}) alters the global context, may be used at application start for example
</span>   </span>;; (u/with-context {:keyN valN} body) macro that will wrap the body in a new, temporary context; more context can be chained together
</span>
</span>  (</span>u/with-context </span>{</span>:username </span>"</span>foo</span>"
</span>                   </span>:handler :some-endpoint</span>}
</span>                   (</span>u/log </span>::business-logic :message </span>"</span>User did something</span>"))
</span>
</span>  </span>;; will log something look this
</span>  {</span>:username </span>"</span>foo</span>"
</span>   </span>:handler :some-endpoint
</span>   </span>:mulog/trace-id </span>#mulog/flake "</span>4q6yXTTvky4yTeHENi8UcRNPpjJrPpPq</span>"
</span>   </span>:mulog/timestamp 1684907603865
</span>   </span>:mulog/event-name :user/business-logic
</span>   </span>:mulog/namespace </span>"</span>user</span>"
</span>   </span>:message </span>"</span>User did something</span>"}
</span></code></pre>
As we have seen in the example, the event has been created with the extra
information set in the call to u/with-context. This is very handy and makes
it easier to have consistent logs across our code.</p>
As we add more context in our call chain we will end up with extremely rich
logs that can be used for analytics. Taking again the example of a web app
we can potentially understand how different parts of our business are being
used by users with different devices using the request user agent or how
many users we have for each country using the request ip, in order to scale
up or down our clusters in those regions.</p>
Traces</h2>
On top of the logging facility, mulog u/trace macro make it possible to measure
the time spent by any part of code wrapped by it, here is its signature:</p>
  (</span>u/trace</span> event-name [</span>:keyN</span> value1N] body)
</span></code></pre>
Like u/log it can leverage nested context data but on top of that it is possible
to introduce new context data that will be captured by the current and nested u/trace
calls but not by nested calls to u/log.</p>
The body parameter is the code that we want to trace. Here is an example call
and the resulting event:</p>
  (</span>u/trace </span>:calling-api </span>[</span>:extra </span>"</span>content</span>"] (</span>post-to-external-api</span> payload))
</span>  
</span>  </span>;; example event logged by the trace call
</span>  {</span>:mulog/duration 250362049
</span>    </span>:mulog/namespace </span>"</span>user</span>"
</span>    </span>:mulog/outcome :ok
</span>    </span>:extra </span>"</span>content</span>"  </span>;; <- here is the extra content provided by trace
</span>    </span>:mulog/parent-trace nil
</span>    </span>:mulog/root-trace </span>#mulog/flake "</span>4q73JVUr9GK_PBOhs6p3NJfnYhqps7Tp</span>"
</span>    </span>:mulog/timestamp 1684953980473
</span>    </span>:mulog/trace-id </span>#mulog/flake "</span>4q73JVUr9GK_PBOhs6p3NJfnYhqps7Tp</span>"
</span>    </span>:mulog/event-name :calling-api</span>}
</span></code></pre>
The example shown here traces within the boundaries of the running process but
mulog can be instrumented to work with distributed tracing systems; for more
details refer to the documentation</a>.</p>
Transformers</h2>
When starting a publisher it is possible to specify a transformation function
that will take a sequence of events to be modified or even evicted, and returns
the new sequence of events. Thanks Bruno for your review</a>!</p>
This approach can be useful to remove sensible data before it is sent to a
log collector, for example user tokens, passwords, API keys and so on.
While we may want to retain this kind of information when developing, and
using a console logger, it is mandatory to remove it while in production
when working with real business data. Another use case is to completely evict
events that we don't want to send to a specific collector, for example we
may have a collector interested in metrics to scale a cluster and another
one dedicated to business logic, the possibilities are endless!</p>
Having the possibility to specify a transformation function at the publisher
level make it easy to configure our systems based on the environment they
are running on.</p>
Closing words</h2>
While it looks simple on the surface, mulog is a powerful library that helps you
to get useful insights about your running system. Fortunately it does not require
a lot of ceremony to get started and can be bent to meet your specific use cases
if needed.</p>
This post is just a brief introduction to the topic of structured logging, viewed
with the lenses of mulog. I highly suggest to dig deeper in its documentation</a> and
watch Bruno's talk "u/log and the next 100 logging systems</a>".</p>
Discuss</h2>

Twitter</a></li>
Reddit</a></li>
Linkedin</a></li>
Fediverse</a></li>
</ul>


Plans to improve the estimation cheat sheet
Unknown — Sat, 20 May 2023 00:00:00 +0000
Project</h1>
Project name: Unrefined</p>
Project URL: https://unrefined.one</a></p>
Project sources: https://gitlab.com/fpischedda/unrefined</a></p>
Cheat sheet redesign</h1>
The current cheat sheet is not ideal for the following reasons:</p>

It is hardcoded and there is no way to chose a different one</li>
It is too specific to one use case/company</li>
The estimation UI with the sliders is not intuitive, hiding the maximum story points</li>
</ul>
Breakdown of tasks</h2>
Change UI to select a cheat sheet from a list of available ones</h3>
When starting an estimation provide the option to select the cheat sheet to be
used during the refinement session from a list of available ones, possibly with
a preview of the breakdown items.</p>
Sub tasks - Size M</h4>

[BE] Provide the list of available cheat sheets to the frontend layer with their details</li>
[FE] Show the pre-selected cheat sheet name</li>
[FE] Send the selected cheat sheet to the backend when starting a refinement</li>
[BE] Use the cheat sheet sent by the frontend when creating a new refinement</li>
[FE] Enable selecting a different cheat sheet from a list of available ones</li>
[FE] Show the breakdown items of the currently selected cheat sheet</li>
</ul>
Change UI of the estimation page</h3>
Explore few different designs for the estimation page to enable new ways of
selecting the story points for each breakdown item. Implement at least one
and provide a way to select between available styles, starting from the
current one. If a user changes the style, set it as default in the local
storage.</p>
Sub Tasks - Size S</h4>

Try a couple of designs before implementing them</li>
[FE] Implement at least one new design</li>
[FE] Enable selecting from available designs</li>
[FE] Set the selected design as the default in the local storage</li>
[FE] Read the selected design from local storage and use that if available otherwise use the default one</li>
</ul>
Customize the cheat sheet</h3>
After selecting a cheat sheet, the user can customize it, creating a new
cheat sheet.</p>
It should be possible to start from a blank page if the current templates
do not match the user's use case.</p>
Evaluate if it is feasible to store the new cheat sheet somewhere, considering
that at the moment there is no way to authenticate a user.</p>
Sub Tasks - Size L</h4>

[FE] Enable cheat sheet customization, adding or removing breakdowns and their items</li>
[FE] Send the custom cheat sheet to the backend when starting a new refinement</li>
[BE] Accept new cheat sheets from the frontend and use them when creating a refinement</li>
[BE] Send the custom cheat sheet to the frontend in estimation and result pages</li>
[FE] Use custom cheat sheets in the estimation and result pages</li>
Evaluate how to store and retrieve custom cheat sheets</li>
</ul>


Build, show & tell
Unknown — Thu, 18 May 2023 00:00:00 +0000
TL;DR Connect with your community! Right, now!!!</h1>
Yesterday, May 16 2023, 19:00 CEST, I've attended a spontaneous
meetup of indiehackers, organized on Twitter by GYN</a>.</p>
Somehow he managed to take a group of strangers, posting under the
hashtag #buildinpublic</a>, and gather everyone together to talk about
what we are building, what kind of help we are after and how we can
help others. And was great!</p>
Here is where it all started</a>.</p>
The meetup</h2>
We started introducing ourselves, our background and our projects,
after that Alexander</a> introduced his projects and I've assisted to
the most honest and insightful round of feedback that anyone can
receive in her/his life by complete strangers!</p>
Honestly it has been amazing, ideas floated around, from UX to
monetizing, to cost analysis and more. And just to repeat, it was
coming from people that have never interacted in real or virtual life!</p>
Closing words</h2>
I wrote this tiny post to encourage everyone to get in touch with their
own community, you'll never know how supportive it can be until you try
it. It is easy to set the boundaries to your (home) office and users'
feedback, but if don't get in touch with other experiencing the same
challenges as yours, I can confidently say that you are missing out.</p>
Please come and join the #buildinpublic</a> community, if don't like it you
lose nothing but I suspect that we have a lot to share and learn from
each other.</p>


Opening wave 2023-05-17
Unknown — Wed, 17 May 2023 00:00:00 +0000
Wave merge request: https://gitlab.com/fpischedda/fp-site/-/merge_requests/9</a></p>
Overview</h1>
Past wave had a bit ups and downs, following the white rabbit of
number of views on this website but not moving the needle of product</a>
awareness, nor investigating other opportunities. A key factor was
(poor) time management, as reported while closing wave 2023-04-26.</p>
Next round</h2>
This wave should focus on the failures of the previous one, trying
to address the issues that have had the most impact on the expected
progress, such as:</p>

Balance: main driving forces are content creation and product development, how to respect both?</li>
Progress: how to make sure energy is spent in meaningful activities</li>
Time management: how to make use of the limited time available</li>
</ul>
Projects</h2>
As always there is work to do for this website, Unrefined and, this
time I am adding a new idea that I'd like to validate.</p>
Website</h3>
Topics:</p>

Clojure Bites: I am enjoying writing this series of posts, realistically I want to have one every two weeks</li>
Project progress: not much happened recently, I want to try to have an update once a week</li>
Project validation: this is a new thing, I have a couple of ideas in mind but I have to scope them well before starting</li>
Video contents: I'd like to record a session about the last Clojure Bites and how to use Unrefined; it is a time consuming activity</li>
</ul>
Unrefined</h3>
I'd like to use some of the feedback I've got recently to move
the project to a state where it is more generally useful and usable:</p>

Estimation cheat sheet: it is an interesting concept that must be generalized</li>
Cheat sheet customization: it is hard to make a generic one and I should enable people to customize it</li>
Estimation breakdown UI: the current slider approach can be improved, I've collected a couple of ideas worth trying</li>
User account + OAuth providers: if users can customize their cheat sheets they may want to keep it with their own accounts</li>
</ul>
Validating a new idea</h3>
A general advice is to build something that solve your problems
with the assumption that others may have the same problem. But
before building the solution is better to validate that there is
enough market for it both in terms of potential users and $$$.
This is the indie hacker/bootstrapper 101, and given I'm a noob
better to start from the basics!</p>
So the plan is:</p>

Find where the potential audience hang out</li>
Understand their needs</li>
Create a landing page and collect potential users</li>
</ul>
I'll reveal the project and results in a future post, until then
stay tuned!</p>
Stats update</h3>
Here is the graph since the beginning of this site, it awesome how
it growed in just one month! Peaks are for the Clojure Bites series
but just because it is easier to target specific communities, and
not because those were great articles :) </p>
</p>
Discuss</h3>
Linkedin</a></p>
Twitter</a></p>


Closing wave 2022-04-26
Unknown — Mon, 15 May 2023 00:00:00 +0000
Overview</h1>
Closing the first wave, I feel the need of a sort of retrospective, pointing
out what went well, what didn't and action items for the next iteration.</p>
TL;DR is that I've probably set ambitious goals for myself (maybe too much),
there are two driving forces competing for my limited time, the content creation
side and the product development side, and I have to find a way to balance
my time budget.</p>
Continue reading for the gory details.</p>
Wave goals</h2>
From the start of the wave</p>

Improving the estimation cheat sheet in Unrefined</li>
Creating a landing page and marketing it (throwing some $$$ in marketing maybe)</li>
Improving this website, there are still some information missing</li>
Reveal the new experiment and first impressions</li>
</ul>
What went well</h2>
Improving the estimation cheat sheet in Unrefined</h3>
This is a mixed bag, on one hand I've received some feedback that the cheat sheet
is a good idea, but in the current form is not that useful. I've started working
on a generic one but it is not an easy task but now it is clear that I can try
to create some default templates and let people customize their own.</p>
Creating a landing page and marketing it (throwing some $$$ in marketing maybe)</h3>
Baby steps, I have added a brief description of how Unrefined works; next step
should be to get a pricing page proposing "Pro" features and measure engagement.</p>
More content and hints about what kind of posts drive more visits</h3>
In this wave I've published two (1</a>, 2</a>)
short articles on Clojure which performed quite well given my small network.
One lesson is that this kind of posts do not perform well on Linkedin, on the other
hand Reddit and Twitter convert pretty well. Also timing is crucial, better to post
early or mid week.</p>
What didn't</h2>
Improving this website, there are still some information missing</h3>
Nothing done yet, plus I wanted to record a screen cast but I've spent too much time
learning OBS and I was not happy with the final result. I have the video still
in my todo list and I want to release it soon!</p>
Reveal the new experiment and first impressions</h3>
No work done here as well. I'm still figuring out the scope of the project which
at this time looks way too big for a single person. I've got other ideas that
I'd like to validate. The biggest blockers are:</p>

Defining scope and core idea</li>
Learning how to quickly build landing pages, it is not a tech problem, more of copy writing</li>
</ul>
Marketing Unrefined</h3>
It was planned to record a video session of how it works, create a landing page
and push it to the wild. None of this happened.</p>
Time management</h3>
Currently I am spending most of my time on content creating and not enough on current
projects, adding meaningful features or potential projects that require some research.</p>
On content creation, I am still finding my way to it and it is time consuming but I suspect
there is a lot of potential there, to drive traffic to products or possibly becoming
a product itself. The general suggestion is to try to find verticals, and this is a whole
big topic to explore.</p>
Action items</h2>
Looking at the broader picture I think I have to:</p>

Get better at managing time: with a full time job every "free" time matters!</li>
Build an operating system that makes sense: connected to previous point; having a process should help with time management as well</li>
Keep ambitious goals, even if I haven't reached most of my goals I think it is better to aim higher than lower</li>
Try out more ideas i.e. get better at pushing out landing pages and looking for early feedback/validation</li>
</ul>
Closing words</h2>
It is important to remind to my(your)self that building something new is hard but
also exciting! There are a lot of unknowns and we learn as we go, so please try to find
the time to look back at what you did, where you want to be and look forward the next
steps.</p>


Clojure bites - dynamically add depencencies at runtime!
Unknown — Fri, 12 May 2023 00:00:00 +0000
Overview</h1>
One pain point of REPL driven development is that even if it is possible to
change an application while it is running, if you want to experiment with a
new library it is need to add a new dependency to your project and restart the
REPL to use it, possibly breaking your flow.</p>
Fortunately a new set of functions to alleviate this pain are in the works</a>
and ready to be tested since Clojure 1.12.0-alpha2</a>:</p>

add-lib: Given a lib that is not yet on the repl classpath, make it available by downloading the library if necessary and adding it to the classloader.</li>
add-libs: Given lib-coords, a map of lib to coord, will resolve all transitive deps for the libs together and add them to the repl classpath, unlike separate calls to add-lib.</li>
sync-deps: Calls add-libs with any libs present in deps.edn but not yet present on the classpath.</li>
</ul>
Current status</h2>
At the time of this post, add-lib is not yet in an official Clojure release
but can be added as a development dependency, possibly putting it in an alias
using for development; it does not make too much sense to have it bundled in
your production build anyway.</p>
Assuming that you have a `:dev/repl` alias in your deps.edn file, you can get
this new functionality by adding version 1.12.0-alpha2 of Clojure to your deps:</p>
  {</span>:dev/repl </span>{</span>:extra-deps </span>{org.clojure/clojure {</span>:mvn/version </span>"</span>1.12.0-alpha2</span>"}}}
</span></code></pre>
add-lib in action</h2>
As an example I am going to re-write the FizzBuzz example from a previous post
to use pattern matching instead of cond.</p>
Initial setup</h3>
First step is to setup deps.edn to use Clojure 1.12.0-alpha2 in a development
alias, it is a best practice to have development namespaces added to the source
path, used to call add-lib, in order to not ship it with the production release.</p>
Another good habit is to have a custom user namespace with code used during
development, usually storing its code in dev/user.clj; to make it available in
the repl, the "dev" directory must be added to :extra-deps vector.</p>
Add the following map to the :aliases section of deps.edn map</p>
  {</span>:dev/repl </span>{</span>:extra-deps </span>{org.clojure/clojure {</span>:mvn/version </span>"</span>1.12.0-alpha2</span>"}}
</span>              </span>:extra-paths </span>["</span>dev</span>"]}}
</span></code></pre>
Put the following content in dev/user.clj</p>
  (</span>ns</span> user
</span>    (</span>:require </span>[clojure.repl.deps </span>:refer </span>[add-lib]]))
</span></code></pre>
Scenario</h3>
For the sake of argument assume that we want to write a new
implementation of FizzBuzz, in order to impress future recruiters.</p>
Looking around we find out that pattern matching</a> is a interesting
approach to conditionals compared to ifs and conds, commonly used in
functional programming languages (for ex. Erlang, Elixir and more) but
also being implemented in more traditional languages like Python
(as of Python 3.10).</p>
Changing FizzBuzz to use pattern matching</h3>
Ready to approach our new mission we fire up our IDE, load the FizzBuzz
project and start a repl, maybe running tests again to ensure that
nothing broke while we were not watching.</p>
Looking around we find core.match</a> and, wow, the first example is an
implementation of FizzBuzz, how lucky!</p>
  (</span>require </span>'[clojure.core.match </span>:refer </span>[match]])
</span>  
</span>  (</span>doseq </span>[n (</span>range </span>1 101</span>)]
</span>    (</span>println
</span>      (</span>match </span>[(</span>mod</span> n </span>3</span>) (</span>mod</span> n </span>5</span>)]
</span>        [</span>0 0</span>] "</span>FizzBuzz</span>"
</span>        [</span>0</span> _] "</span>Fizz</span>"
</span>        [_ </span>0</span>] "</span>Buzz</span>"
</span>        </span>:else</span> n)))
</span></code></pre>
Two things to notice here:</p>

This does not match the function signature that our tests expect</li>
Anyway this code snippet will fail because core.match is not in the classpath</li>
</ul>
Instead of adding the dependency to deps.edn and restarting the repl, we can
download this dependency just for this session using add-lib.</p>
From user namespace we can evaluate the following form to download the new
library to try it out:</p>
  (</span>add-lib </span>'org.clojure/core.match)
</span></code></pre>
At this point the repl will show all the depencencies being downloaded to
satisfy our request. Once done it will possible to require the new library
and use it:</p>
  (</span>require </span>'[clojure.core.match </span>:refer </span>[match]])
</span>
</span>  (</span>match </span>[</span>true false</span>]
</span>    [_ </span>true</span>]     </span>:case1
</span>    [</span>false</span> _]    </span>:case2
</span>    [</span>true false</span>] </span>:case3</span>) </span>;; => :case3
</span></code></pre>
Beautiful! We have added a new lib to our running repl, we have tested
it and we are ready to use it to solve our problem.</p>
Here is the new fizzbuzz implementation using core.match</p>
  (</span>defn </span>fizzbuzz </span>[n]
</span>    (</span>match </span>[(</span>mod</span> n </span>3</span>) (</span>mod</span> n </span>5</span>)]
</span>      [</span>0 0</span>] "</span>FizzBuzz</span>"
</span>      [</span>0</span> _] "</span>Fizz</span>"
</span>      [_ </span>0</span>] "</span>Buzz</span>"
</span>      </span>:else</span> n)
</span></code></pre>
Does it work? Well there is a proper test suite so lets try it out…and it works!</p>
</p>
Closing words</h2>
It does not happen every day to want to try a new library when working on something
but at times I really really wanted to have a new dependency being added to a running
project, without losing the current state, context and focus. And it is extremely
simple, so give it a try!</p>
Further reading</h2>
In this post I skimmed trough a lot a topics to focus on the add-lib
flow, so here are some resources worth reading if you want to dig deeper,
and I highly suggest so!</p>

Praticalli</a> on hot reloading libraries</a> and his clojure-cli-config</a> project (plus tons of awesome documentation)</li>
clj-new</a> to create new projects using Clojure cli, not specifically mentioned in the post but useful</li>
core.match docs</a>, in you want to dig deeper (highly suggested)</li>
</ul>
Discuss</h2>
Looking forward your feedback!</p>

Twitter</a></li>
DEV</a></li>
Reddit</a></li>
Linkedin</a></li>
</ul>


Unrefined generic cheatsheet
Unknown — Sat, 06 May 2023 00:00:00 +0000
Project</h1>
Project name: Unrefined</p>
Project URL: https://unrefined.one</a></p>
Project sources: https://gitlab.com/fpischedda/unrefined</a></p>
Overview</h2>
Unrefined's UX for estimations is centered on the estimation cheatsheet,
with the assumption that it would help engineers to breakdown tasks to
smaller items and assign a number to each of them.</p>
A cheatsheet provides a list of breakdown items such as:</p>

Implementation</li>
Testing</li>
Migrations</li>
Risk</li>
...and more</li>
</ul>
For each item there is a description of the possible actions for example,
considering implementation:</p>

Small change to an existing functionality: 1 story point</li>
New endpoint for an existing domain model: 3 story points</li>
Implement two or more domain models with their endpoints: 8 story points</li>
</ul>
With the guidance of the cheatsheet, over time it will become easier and 
more predictable how to estimate a task, getting to consensus much faster.</p>
One thing it does NOT is to make estimations more ACCURATE! Estimations are
gut feeling predictions of how much it could take to complete a task, and
should NOT be used to set deadlines.</p>
Here is how it looks in Unrefined's estimation section</p>
</p>
The problem</h2>
The current implementation comes from our experience at $work so it is
very specific to our needs, making not so useful for other team in a 
different domain, with different tech stacks and so on.</p>
For this reason I have started to abstract away the breakdown items and 
their description but it is clearly very hard to come up with something
that can capture every possible permutation of stacks, habits, business
domain and what not. This is a source of frustration because it makes
it harder to market the tool to other teams.</p>
Going forward</h2>
I am not sure how to proceed with the cheatsheet but I have some ideas:</p>

Create a generic one, knowing that it will not be a perfect fit for all cases</li>
Start collecting feedback and create specialized cheatsheets</li>
Give people the opportunity to create their own cheatsheet</li>
</ul>
I think that all of the above options are viable and would help to make this
tool more useful to a broader audience, maybe even outside the software
engineering circle.</p>


Clojure bites - clojure.test/are
FPSD — Tue, 02 May 2023 00:00:00 +0000
Overview</h1>
Clojure's clojure.test/are</a> provides a data driven approach to unit testing.
Lets start with a practical example, implementing the most important function
in the history of computer science, FizzBuzz! (But only because all of my
binary search trees are already balanced).</p>
FizzBuzz</h2>
It is (was?) common that, during an interview, to be asked to implement the
logic of the FizzBuzz game, Wikipedia has a nice article</a> about it.</p>
It can be summarized as follows:</p>
Write a function that takes a numerical argument and returns:</p>

The string Fizz if the number is divisible by 3</li>
The string Buzz if the number is divisible by 5</li>
The string FizzBuzz if the number is divisible by both 3 and 5</li>
The argument if none of the previous conditions are met</li>
</ul>
The implementation</h2>
Lets start by defining the test suite using the usual clojure.test/is macro.</p>
  (</span>ns</span> fizzbuzz.core-test
</span>    (</span>:require </span>[clojure.test </span>:refer </span>[deftest is testing]]
</span>              [fizzbuzz.core </span>:as</span> sut]))
</span>  
</span>  (</span>deftest </span>OMG-FizzBuzz
</span>    (</span>testing </span>"</span>Should return the numerical argument</span>"
</span>      (</span>is </span>(</span>= </span>1 </span>(</span>sut/fizz-buzz </span>1</span>))))
</span>  
</span>    (</span>testing </span>"</span>Should return Fizz</span>"
</span>      (</span>is </span>(</span>= </span>"</span>Fizz</span>" (</span>sut/fizz-buzz </span>3</span>))))
</span>    (</span>testing </span>"</span>Should return Buzz</span>"
</span>      (</span>is </span>(</span>= </span>"</span>Buzz</span>" (</span>sut/fizz-buzz </span>5</span>))))
</span>    (</span>testing </span>"</span>Should return FizzBuzz</span>"
</span>      (</span>is </span>(</span>= </span>"</span>FizzBuzz</span>" (</span>sut/fizz-buzz </span>15</span>))))
</span>    )
</span></code></pre>
Please note that sut stands for system under test; I've seen it being used
here and there but I am not sure it is a best practice or not.</p>
The test will clearly fail because there is no fizz-buzz function or even a
fizzbuzz.core namespace. Lets start with a trivial implementation.</p>
  (</span>ns</span> fizzbuzz.core)
</span>  
</span>  (</span>defn </span>fizz-buzz </span>[n]
</span>    (</span>cond
</span>      (</span>= </span>0 </span>(</span>mod</span> n </span>15</span>)) "</span>FizzBuzz</span>"
</span>      (</span>= </span>0 </span>(</span>mod</span> n </span>3</span>)) "</span>Fizz</span>"
</span>      (</span>= </span>0 </span>(</span>mod</span> n </span>5</span>)) "</span>Buzz</span>"
</span>      </span>:else</span> n))
</span></code></pre>
Now all tests are passing, the interviewer is more than happy but you
want to show off your skills and ask to improve both code and tests</p>
Improvements</h2>
First thing to notice is that if a number is not a multiple of 3 or 5 then
we run 4 divisions and return n. A slightly improvement can be the following:</p>
  (</span>ns</span> fizzbuzz.core)
</span>  
</span>  (</span>defn </span>fizz-buzz
</span>    [n]
</span>    (</span>cond
</span>      (</span>= </span>0 </span>(</span>mod</span> n </span>3</span>)) (</span>if </span>(</span>= </span>0 </span>(</span>mod</span> n </span>5</span>)) "</span>FizzBuzz</span>" "</span>Fizz</span>")
</span>      (</span>= </span>0 </span>(</span>mod</span> n </span>5</span>)) "</span>Buzz</span>"
</span>      </span>:else</span> n))
</span></code></pre>
Test are passing so we are confident that the function is working as expected,
and it is a bit more performing! Yes, we are not solving the world's energy
crisis but it is something.</p>
Data driven tests</h2>
Looking at the tests we can notice that we are calling the same function with
different input values and expecting a specific result. User of other testing
libraries, for example Pytest</a> may be familiar with the parametrize</a> decorator
that takes tuples of data and calls the test case with that data as parameters.
In Clojure we can achieve that with clojure.test/are macro, here is the docstring:</p>
"Checks multiple assertions with a template expression.
See clojure.template/do-template for an explanation of
templates."</p>
A bit cryptic but an example can help us understand better.</p>
  (</span>ns</span> fizzbuzz.core-test
</span>    (</span>:require </span>[clojure.test </span>:refer </span>[deftest are]]
</span>              [fizzbuzz.core </span>:as</span> sut]))
</span>
</span>  (</span>deftest </span>OMG-FizzBuzz
</span>    (</span>are </span>[argument expected] (</span>=</span> expected (</span>sut/fizz-buzz</span> argument))
</span>      </span>1 1
</span>      </span>2 2
</span>      </span>3 </span>"</span>Fizz</span>"
</span>      </span>6 </span>"</span>Fizz</span>"
</span>      </span>5 </span>"</span>Buzz</span>"
</span>      </span>10 </span>"</span>Buzz</span>"
</span>      </span>15 </span>"</span>FizzBuzz</span>"))
</span></code></pre>
And voila, we have a data driven test suite for our implementation!</p>
Is it all rainbows and unicorns?</h2>
clojure.test/are comes with few shortcomings:</p>

Reporting: if one of the cases will fail, the report will not point directly to offending line but the line where are is called; usually it is not hard to get to the failing case anyway</li>
It is easy to put a call to clojure.test/is inside a call to are</em>, this will make the pass anyway; generally speaking a new test case should fail at first and then we adjust it to make it pass, this should protect for the accidental use if is</em> but it is something to keep in mind.</li>
</ul>
Alternatives</h2>
clojure.test/are is a builtin macro but it comes with some problems (not so big IMHO), so are there valuable alternatives? Some lovely people pointed them out in the comments
of the post</a> on Reddit:</p>

plain doseq: the rationale is that are</em> introduce new syntax where doseq</em> would do the same if massaged correctly</li>
humane-are</a> a drop-in replacement for are</em> with benefits such as better error reporting and bindings check (my personal favourite)</li>
</ul>
Closing words</h2>
I hope this will encourage exploring Clojure's core library, to spot little
gems like this one, and to have added a new tool to your toolbox!</p>
Code for this post can be found here</a>.</p>


Random bits
Unknown — Thu, 27 Apr 2023 00:00:00 +0000
Topics</h1>
This short post will cover:</p>

guile-commonmark bug</li>
website next steps</li>
OBS</li>
experiment</li>
</ul>
The bug that almost got me out of Haunt static site generator</h2>
At the time of the second post I was already super excited by the
project, the content was coming out nicely and wanted to have it 
out in the wild ASAP! As usual I launched haunt build</code> command to
generate the static pages and all I got was an exception...my mood
went down pretty quickly.</p>
I was sure the problem was on my side, probably something formatted
the wrong way but I could not make it to work. I was thinking it was
a bug in Haunt itself but after carefully inspecting the stack trace
I've tracked down the problem to guile-commonmark</a>,
for which I've opened an issue</a>,
hoping to get some feedback soon(ish). Unfortunately I've quickly realized
that the project is not actively developed, and decided to find a 
workaround for it. Even worse, the test suite is failing so it is not
easy to understand if I am breaking other places.</p>
At some point I've identified the line that was breaking and commented
it, only because it was doing some cleanup, and having some extra 
spaces or unneeded HTML tags in the output is not the end of the world.</p>
If curious or you are affected by the same problem, this is the line to
comment link</a>.</p>
p.s. I think this happens because I am compiling with Guile 3.0.x, maybe with 
Guile 2.x it works well...but I haven't tested it yet.</p>
Website next steps</h2>
I want to keep the website features to the bare minimum but at the same
time there are some missing bits:</p>

Links to contacts and social accounts</li>
Project pages</li>
Tags</li>
Maybe a page for the CV?</li>
</ul>
I will slowly work on these parts, there is no rush but at least it is written
somewhere publicly and not only in my "secret" project files :) .</p>
OBS</h2>
I've started learning to use OBS because I want to record a couple of videos to
explain how Unrefined</a> works and how the Chrome Extension
may improve the workflow.</p>
I've got a course on Udemy</a>
and I am liking it a lot so far! (Not affiliated).</p>
Frankly speaking, one day I would like to try to stream a coding session on one of
my projects to get live feedback and have some fun! I am an introvert and I am 
postponing this step every week, but at some point it will happen, so stay tuned!</p>
The experiment</h2>
This time I'll limit the number of places where I'll advertise my posts in order to
understand how much recurring traffic I have, if any. Clearly marketing helps a lot
but at times I feel I am a bit spammy and I am considering to promote fewer posts
in future. Lets see how it goes :) .</p>


Wave start - 2022-04-26
Unknown — Wed, 26 Apr 2023 00:00:00 +0000
Goals</h2>
In this wave (yes, yes, wave sound better than sprint) I'll work to make some
progress on Unrefined</a> and, for the first time, I will
try to launch something new which I have in the back burner for some time.

For the first time I will be trying to use the most suggested approach to build
a landing page and see if there is interest in the product, before jumping to the
code. It would be great to collect some pre-sales but I don't know to deal with
them given that I don't have a legal entity yet, and I don't want to get in
troubles with the law...</p>
Roughly I am going to spend my time on:</p>

Improving the estimation cheatsheet in Unrefined</li>
Creating a landing page and marketing it (throwing some $$$ in marketing maybe)</li>
Improving this website, there are still some information missing</li>
Reveal the new experiment and first impressions</li>
</ul>
See you in the next update, and don't forget to subscribe to the RSS</a> feed!</p>
For live updates track this MR</a></p>


Closing sprint 2022-04-17
Unknown — Tue, 25 Apr 2023 00:00:00 +0000
Project name: FPSD</p>
Project URL: https://fpsd.codes</a></p>
Project sources: https://gitlab.com/fpischedda/fp-site</a></p>
Sprint merge request: https://gitlab.com/fpischedda/fp-site/-/merge_requests/6</a></p>
How it went</h2>
Sprint 2022-04-17 has been quite different from the first one, as I've already
figured out some details on how to run this web site I have spent more time 
on the next steps for my projects. This is a good outcome because I was worried
that I was spending too much time on marginal topics instead of making progress
on my projects.</p>
Finally I have released</a> the Chrome Extension
for Unrefined</a>; I was procrastinating this release for
too long and after mentioning it in a previous post I was feeling the pressure
I've put on myself which gave the energy boots to finally make it public.

It is not perfect yet but at least I can start to get some feedback!</p>
Numbers</h2>
I was trying to get to 100 unique visitor before making this post but, honestly,
having reached 92 is already a good result for something started 15 days ago,
and basically no network!</p>
Lets have a look at what Plausible says:</p>
</p>
Key takeaways:</p>

views are still too bounded to my post on socials, no recurring traffic</li>
longer posts have better performance</li>
(Not visible from the graph) Still no engagement</li>
Some sources are better than others, in order: Linkedin, Twitter, DevTo, fediverse</li>
Copy/Pasting the same post to all social account does not work well, especially on DevTo</li>
</ul>
Closing words</h2>
There is still a lot to learn but I am enjoying the experience so far. Being an introvert
it makes it hard to push a bit more on the marketing side, but in the Internet
no one knows that you are a dog ;) .</p>
One last thing, I will discontinue the term "sprint" in favor of wave; to me it feels
more in line with what I am doing and my mental processes.</p>


Announcing Unrefined's Chrome Extension
Unknown — Mon, 24 Apr 2023 00:00:00 +0000
Project</h1>
Project name: Unrefined</p>
Project URL: https://unrefined.one</a></p>
Project sources: https://gitlab.com/fpischedda/unrefined</a></p>
Extension sources: https://gitlab.com/fpischedda/unrefined/-/tree/main/browser-extension</a></p>
Chrome Extension for EMs is out!</h2>
After a bit of testing, the new Chrome Extension is out, targeting Unrefined's
production environment!</p>
What was stopping its release was that I had no build script in place to package
the extension, and asking to install it by pulling the repo and loading it
manually is a bit too much, even for Engineering Managers.</p>
It is still not an "official" extension so, to work, it must be installed using
developer mode. I am planning to put it in the Extensions Store at some point but
better to wait until it will have a least few rounds of tests.</p>
Here</a> is how to install an extension in developer mode.</p>
You can get it from this link</a>. It will be in Unrefined's header after the next release.</p>
How it works</h2>
After installing it, a user can start and manage a refinement session directly
from the extension's UI, without going to Unrefined web application.</p>
Each time a new ticket is refined, a link to the estimation page is copied to
the clipboard so that the EM can share this link with the Engineers; actually
this is needed just the first time a refinement session is started, when
estimating new tickets, the estimation pages will update automatically, using
SSE. It may still be helpful to share the link again, in case someone else
will join an already started session, or if someone closes their browsers.</p>
I am planning to record a couple of videos to show a session with and without
the use of the extension. (Never done that, it will be a mess for sure…)</p>
Build script</h2>
A few words about the build script; I needed something extremely simple to:</p>

switch to production settings</li>
package everything in a zip file, for easier distribution</li>
</ul>
Another requirement for me was to try to not introduce other tools for such a
simple task.</p>
Fortunately the web application is build using tools.build</a> which is the
official, builtin, build tooling provided by Clojure, so the requirement
to not reach out to other tools has been satisfied!</p>
It required literally few minutes to hack a target for the extension, which
does just what I need:</p>

clean previous build directory if any</li>
copy the extension sources to a build directory</li>
switch to prod settings</li>
remove unneeded files</li>
create a zip archive</li>
</ul>
The API</a> of tools.build library is simple and easy to reason about, plus there
are many example out there from which to get "inspiration".</p>
Next steps, just QoL improvements for my release process:</p>

build the extension when building the application's jar</li>
copy new artifacts to the production environment</li>
switch to the new version of the app and extension</li>
</ul>
I am still not using any containerized environment and I am postponing this step
for when there will be a real benefit; so far my small VPS is more than enough
to run this project (and a couple of others BTW).</p>
Reference change set</h2>
To get an idea of the changes required for the build script head over to this
MR</a>.</p>
Closing words</h2>
I think that the Chrome Extension is a nice quality of life improvement for EMs
using Unrefined. I am eager to get feedback on it and work on its next iterations,
there is a lot of potential there!</p>
Remember to subscribe to the RSS</a> feed to get updates on my progress, and stop by
to say "hello" on my social media accounts! Yes those are not linked here yet, but
chances are that you are getting to this page from my posts ;) </p>


New style using Terminal CSS
Unknown — Sat, 22 Apr 2023 00:00:00 +0000
From 0 to 🤩</h1>
I've started this website not more than 10 days ago (at the time of this post), and

since then it had no styling, just bare HTML, a list of posts and a link back to HOME.</p>
While it was not a problem for me in general, at the same time I wanted to start putting

some links here and there, for example in a nav-bar at the top.

Instead of doing everything from scratch as I usually do, I decided to take a look around

and from all the available options and usual suspects I've found Terminal CSS</a>.</p>
What I like about it:</p>

minimal styling</li>
easy to implement, just inspect the elements on its website and you are ready to go!</li>
very small size ~3k gzipped</li>
</ul>
From now on it will be my go to CSS framework, well done Terminal CSS team!</p>


Updates on projects
Unknown — Fri, 21 Apr 2023 00:00:00 +0000
Updates</h1>
What's going on</h2>
Hi there!</p>
This sprint started in a different way, compared to the previous one.</p>
One key difference is that I have published less content, the reason

being that I've been busy at work and I haven't been able to focus on

new posts, even if I had a couple of ideas ready:</p>

A bug in the guile-commonmark library used by Haunt and I have worked around it</li>
Work on the site layout and describe how to achieve it with Haunt</li>
Next steps for Unrefined</li>
</ul>
The size of the posts would have been micro, small and medium respectively,

so quite achievable compared to the previous week. So what happened? </p>
Getting back to the roots</h2>
I have started this site to document the progress of my projects and I have realized

that, excluding the site itself, I haven't improved any other project so far, and

not because there is not any work to do!</p>
To be honest, it was expected that it requires some time to bootstrap a website,

but at the same time content about the projects is king here, and I want to give

it the top priority. In order to get more content about my progresses I have to

actually work on the projects! :D</p>
Unrefined's backlog and next steps</h2>
Today I had a chance to get back to Unrefined, filling up its backlog and

prioritizing it based on the feedback I've received and the experience

accumulated using it at work.</p>
Following an extract of the project's backlog:</p>
Browser extension</h2>
Goal of the browser extension is to streamline the flow of the refinement

session for both EMs and Engineers.
For EMs:</p>

refine a ticket from the tickets page without moving to Unrefined home page</li>
get links for estimations and updates</li>
view estimation results</li>
re estimate the current ticket</li>
estimate a new ticket</li>
</ul>
For Engineers:</p>

join an ongoing refinement session</li>
estimate current ticket from the ticket's page</li>
jump to the next ticket's page</li>
</ul>
The EM user flow is already implemented it needs some testing and packaging

the extension for production use. For now the extension works only on

Google Chrome/Chromium, other browsers will be addressed in future.</p>
The Engineer user flow is yet to be implemented.</p>
Priorities</h3>


TODO Implement a build system for the extension, package and distribute it</p>
Breakdown of the build system:</p>

Generates a zip archive with the production version of the extension</li>
Generates a version of the extension that does not need developer mode</li>
Publishes the extension to the store</li>
</ul>
</li>

TODO Design the Engineer user flow</p>
</li>

TODO Implement the Engineer user flow</p>
</li>

TODO Launch and beta test</p>
</li>
</ol>
Generic estimation cheat sheet</h2>
The estimation cheat sheet should help Engineers to breakdown the work needed

for a ticket and assign to each item the appropriate story points. The sum of

all items' estimations will give the story point for the ticket being refined.</p>
Another benefit of the cheat sheet is that it creates consistency across estimations,

in case of outliers it is easy to find out if someone over/under estimated the

effort and makes it possible to discuss the different views on the ticket being

refined.</p>
Consistency also helps Product teams to forecast the effort needed to implement

new features by looking at past, similar, tickets' estimations.</p>
Currently the cheat sheet is hard coded and specific to a team for which this tools

has been built. Other team may need different entries and, thinking a bit further,

could be applied out of the software engineering world.</p>
Priorities</h3>


TODO Provide a second, more general, hard coded cheatsheet</p>
</li>

TODO Provide a way to customize the cheat sheet</p>
</li>
</ol>
Coming next</h2>
There are few more entries in the backlog which are not well defined yet:</p>

Admin panel: to at least have a view of what is going on in the web app</li>
Organizations and users: there is no concept of user yet, everything is publicly accessible</li>
Stats: once users and organizations will be in the system it might be valuable to have some stats, yet to be verified</li>
</ul>
The plan is to apply the final touches to the browsers extension and later focus on

the estimation cheat sheet which, according to the feedback, is one of the most valuable

features of the whole system, together with the estimation flow.</p>
I am tempted to try to work on the next features during a live stream. The idea is

intriguing but at the same time terrifying, I've never streamed before so it could

be a fun experience as well as a terrible one!</p>
Closing words</h2>
It has been a busy week, I have produced much less content than expected but at the same

time I needed a break to think about how to structure this effort to build in public.</p>
There are still many lessons to learn, one of which is how to use the limited time I have

at hand for my side projects. I hope to find a way to make this experience enjoyable for

everyone (me included ;) ).</p>
Stay tuned for the next updates!</p>


Who is using Unrefined?
Unknown — Tue, 18 Apr 2023 00:00:00 +0000
Who is using Unrefined?</h1>
Excluding my colleagues at my day job I don't know!</p>
Recently I have shown Unrefined</a> to the world but I have no idea if

someone else is using it. Yes I have "some" logs but those are not really useful

to track visits, more for debugging purposes.</p>
Given that I am liking Plausible</a> so much I have

added their analytics script to Unrefined</a>.</p>
As always, it is as easy as adding a script tag, here</a> is the change.</p>
That's it for today, as always feel free to share feedback or just say hello :)</p>


New sprint started!
Unknown — Tue, 18 Apr 2023 00:00:00 +0000
[2023-04-18] A new cycle of post is starting</h1>
I enjoyed my first week with this new blog!</p>
The pace is almost right, I don't feel overwhelmed by my expectations,

and I think I can find the time to write a bit more.</p>
Priority for now is to get back to Unerfined and finish some pending

work regarding:</p>

Analytics: add Plausible, yes I have other logs but are not nearly as useful</li>
Improve the cheatsheet, it is too specific to my employer needs</li>
Finish and release the browser extension, potentially targeting more browsers</li>
</ul>
On the micro posts side I have not much at hand yet:</p>

Few words on a bug in the guile-commonmark library</li>
Updates on the site analytics and making them public</li>
Adjustments to the site's layout, linking to my social accounts</li>
</ul>
As expected, the source material will be written
in docs</code> directory and tracked in a fresh new branch and MR so that I can receive comments
and suggestions, the MR is available here</a>.</p>
Feel free to stop by, even to just say hello :) .</p>


Closing first sprint
Unknown — Mon, 17 Apr 2023 00:00:00 +0000
How it went</h1>
In this shot post I want to highlight what happened and first impressions about
the process.</p>
The process</h2>
The idea is to break the development of the website in cycles of one or two weeks.</p>
During this time I've started collecting material about my projects, ideas for posts

and setting up the development tools and flow.</p>
Each cycle I should produce at least one big or medium sized post about a side project

and at the same time I should write micro posts to add new material to the site and

get more visits.</p>
Micro posts worked really well to keep the creative flow and attracting new visits

to the site. Main topic of the micro posts was Haunt, the static website generator

and I suspect that talking about tools in this bite sized format works really well.</p>
How it went</h2>
I would say not bad for a new website with no audience other than my personal network

in the various social platforms, I may also have acquired a couple new followers,

which is really nice!</p>
Few stats:</p>

total unique visitors: 36 (39 - my views from different browsers)</li>
average visitors per day: 6</li>
unique visitors for the longer sized post: 20</li>
</ul>
I would say that the site acquired 5 new visitors each day which is good, at least for

my little experience running a site, not really a scientific analysis but enough for

this stage.</p>
Next</h2>
This post closes the first sprint started the 2023-04-09.</p>
It took a little longer a week, I was expecting it take a bit longer, and

probably for deep dives and more content rich posts it will take that much.

For this reason I am not setting an hard deadline for each sprint, at least

until I'll find a reasonable benefit by doing so.</p>
I'll start the new sprint (2023-04-17) later today.</p>


Introducing Unrefined
FPSD — Sun, 16 Apr 2023 00:00:00 +0000
Overview</h1>
Project name: Unrefined</p>
Project URL: https://unrefined.one</a></p>
Project sources: https://gitlab.com/fpischedda/unrefined</a></p>
What is this about</h2>
Unrefined aims to help remote software engineering teams to run ticket refinement
and estimation sessions.</p>
Refinement sessions are usually attended by:</p>

PMs: presenting their tickets and looking for estimations to plan next sprints</li>
Engineers: refining tickets and providing estimations for them</li>
EMs: coordinating the whole session</li>
</ul>
Unrefined mainly targets EMs who coordinate the refinement and estimation sessions,
and Engineers who are in charge to break down the ticket in its actionable parts
and give an estimation of the effort needed to implement them.</p>
Why</h2>
I've started Unrefined essentially to scratch my own itch. At work we tried different
approaches and all of them never worked properly. Only one was really cool, a Zoom
bot developed by one of my colleagues, but at some point we left Zoom and I missed it
a lot. After that we used to cast our estimations in the chat of the video conferencing
tool we switched to. And that had few problems:</p>

Estimation bias: sometimes people wait for other votes to express their own</li>
A bit hard to coordinate and to find out the final estimation</li>
</ul>
Wh I liked about the bot:</p>

Everyone's estimation was secret so there was no bias</li>
The bot spitted out the final estimation or pointed out if more discussion was needed</li>
Easy to use</li>
</ul>
Development process</h2>
Initial work</h3>
Based on the features provided by the bot, I have started thinking about how to structure a
web app or an api to manage these sessions, the number of features started growing
without even thinking at the core logic: provide a way to express estimations and give
out a result (either an estimation or a suggestion to discuss more).</p>
At this point I've decided to scratch all of the unneeded features and started to implement
the core logic, without even thinking about how people will interact with it.</p>
I have opted for Clojure</a> because I like its interactive,
iterative development flow, being able to shape the program while it is running,
getting a taste of how the API would be used while using it.
There are many more reason to choose Clojure for your projects but this is not the right place to talk about that.</p>
During a lazy Sunday I have quickly sketched out the core, which can be seen at this
commit</a>.

(Yeah, the commits are almost always a mess…but I'll get disciplined later)</p>
At that point I was so excited by its simplicity that I wanted to provide a way
to actually use it!</p>
Making it usable</h3>
Having built the foundations it was time to think about how people would use it.</p>
Goals:</p>

Easy to access and use, people should be able to create and join sessions with the least effort possible</li>
Easily get to the ticket details by linking to the team/company ticketing system</li>
Avoid estimation bias by hiding all estimations until the final result is shown</li>
Instantly get the estimation result</li>
Easily estimate a new ticket if any</li>
</ul>
This is the flow that I came up with:</p>

EM puts the ticket URL in the home page form field</li>
EM gets the estimation link for engineers and shares it</li>
Engineers discuss the ticket and express their estimation in the estimation page</li>
EM + Engineers get realtime updates of the estimations currently expressed (now there is no bias)</li>
EM gets the final estimation, updates the ticket with the resulting story points</li>
EM starts a new estimation for the next ticket if any</li>
Engineers browsers will update automatically to the next estimation page</li>
</ul>
This flow stayed the same since the beginning and it worked quite well, at least for our needs at work.</p>
One feature that came after the first draft is the estimation cheatsheet.</p>
Estimation cheatsheet</h3>
Our estimation cheatsheet is a guideline that helps everyone to stay in the same page
when expressing the estimation and improve coherence, but this does not mean that
estimations are always equal because someone may include effort not considered by others like test, migrations and so on.</p>
How the estimation looks like? I proposes a breakdown of activities, for example:</p>

implementation: code changes from small ones like adding a field to a response JSON or bigger ones like adding a new endpoint</li>
automated tests: how many test should be added? If there is some tech debt test may require more effort</li>
database migrations: are we just adding/removing one field? Is a data migration needed?</li>
</ul>
…and some more entries, but you get the point.</p>
Adding the cheatsheet to the estimation page helped a lot to have more predictable and
coherent estimations. This is helpful for the product team too because, by looking at
past tickets they my have a better idea of the effort required by a new feature.</p>
Moving forward</h3>
Unrefined is now actively used at my workplace and I am proud of the outcome and I think
it could be implemented in other organizations. But before that I want to generalize the
estimation cheatsheet and provide a way to create a custom one.</p>
Another feature I have in progress is the browser extension to help EMs to run estimation session. The extension is almost there but it needs some more testing.</p>
Next steps [0/3]</h3>
1 TODO improve the cheatsheet management</p>
2 TODO finish the browser extension, test it and port it to more browsers</p>
3 TODO try to get feedback from other organizations or engineering teams</p>
Feedback please</h3>
In case you want to share some feedback please come over to the project repo</a>, it is open to everyone! </p>


Customizing Haunt's base layout
Unknown — Wed, 12 Apr 2023 00:00:00 +0000
[2023-04-12] Learning how Haunt's theme/layout system works</h1>
I just want to embed a script!</h2>
Yesterday</a> I've integrated plausible.io</a>
into the website. The process is extremely simple, like in most cases it is as easy
as adding a script tag to embed the tracking code and you are done.</p>
Most static site generators usually have a set of template HTML files that are
filled with posts index, post contents and other stuff, but Haunt is a bit different,
as it provides its own template language that in turn will generate the HTML code.
The DSL reminds me a bit Hiccup</a>, just replace
vectors with lists and keywords with symbols and the two are basically the same...if we 
don't focus too much on the details.</p>
This is the first time I am using Haunt and I had to do a bit of digging to figure out
how to add the tracking script. Fortunately there are plenty of examples, starting
from the project's documentation sources</a>,
where it is possible to see how the author is embedding</a>
piwik analytics in the project's website.</p>
Other site built with Haunt (together with sources) are listed here</a>.</p>
Site configuration file</h2>
Haunt expects a configuration file called haunt.scm, with the settings of the site like
domain name, site title, where to find the source posts etc in order to be able to build the
final site. Part of the settings are which readers to use, for example CommonMark</a>,
or builders such as blog, feeds, static assets etc.</p>
Each builder is customizable and the blog</code> builder accept a #:theme</code> parameter where it is
possible to provide a custom theme; when not specified the builtin
ugly-theme</a> is used.</p>
The theme is declared using the theme</code> function that accepts the following parameters:</p>

name: name of the theme.</li>
layout: a function that generates the HTML layout of the site.</li>
post-template: a function that generates the HTML template of a single post.</li>
collection-template: a function that returns the HTML template to represent a list of posts.</li>
pagination-template: a function that returns the HTML template used for the pagination of posts.</li>
</ul>
Fortunately each parameter is optional so, to just customize the general layout to
include the script for the tracking code, it is enough to create a new function,
overriding the base layout function, to include the custom code and that's it! Ez Pz</p>
Here is my current custom theme in all its glory</p>
(define (fpsd-default-layout site title body)
</span>  `((doctype "html")
</span>    (head
</span>     (meta (@ (charset "utf-8")))
</span>     (script (@ (type "text/javascript")
</span>		(defer "true")
</span>		(data-domain "fpsd.codes")
</span>		(src "https://plausible.io/js/script.js")))
</span>     (title ,(string-append title " — " (site-title site))))
</span>    (body
</span>     (h1 ,(site-title site))
</span>     ,body)))
</span>
</span>(define fpsd-theme
</span>  (theme #:name "FPSD"
</span>         #:layout fpsd-default-layout)) ;; <- please notice that I have defined only the layout
</span></code></pre>
Conclusion</h1>
It is entertaining to play with new tools, sometimes we expect them to work in a certain
way and it can be frustrating if they don't but, if we spend enough time trying to
get to understand them, then we can learn something new and even have some fun time!</p>


Adding analytic
Unknown — Tue, 11 Apr 2023 00:00:00 +0000
[2023-04-11] I need analytics</h1>
What is the point to build in public if I have no idea if my projects,
including this website, are getting traction?</p>
I have no idea of the best practices, tools and whatever is needed to
keep track of traffic, conversion rates, retention and all the rest
so I have no clear idea to what look for, I guess I will learn along
the way.</p>
Looking around for tools I have decided to try plausible.io</a>
because it is simple and has an eye on user's privacy which is a good
bonus to have.</p>
It offers an on premise and a hosted solution which is reasonably priced;
even if at first I was going to try the self hosted option, at 9 EUR/Month
it is not a huge investment and I can get started right away, so lets click
the "Start free trial" button and see what happens.</p>
Another neat feature is that the dashboard can be publicly accessible, I am
considering opening them starting with the website analytics, at least it
looks intriguing.</p>


Starting work on next post
Unknown — Sun, 09 Apr 2023 00:00:00 +0000
[2023-04-09] Started collecting material for a new blog post</h1>
According to my plans I am supposed to create a blog post announcing that I am starting
to work on a new post, so here we are.</p>
As expected, the source material will be written
in docs</code> directory and tracked in a fresh new branch and MR so that I can receive comments
and suggestions, the MR is available here</a>.</p>
Feel free to stop by, even to just say hello :) .</p>
The next post should be a presentation of the projects I am working on but I am not sure 
if I will focus on a single project or if the post will be an overview of what is 
currently in the pipeline.</p>
Another aspect that I want to tune is how much I should say in the sprint start post,
for today I am keeping it simple.</p>


Hello, World!
Unknown — Sat, 08 Apr 2023 00:00:00 +0000
Hello, World!</h1>
My name is Francesco, sometimes you can find me by the handle fpsd</code> around the 
net, sometimes with other handles but probably used for others purposes.</p>
In this website I will collect brain dumps around programming, software/product
development or any other topic I'll find worth sharing, even if, sometimes, not
so worth reading ;).</p>
Soon I'll share my goals and expectations for this project, together with its 
development plan, so stay tuned!</p>
This site was generated by the awesome, hackable, site generator
Haunt</a>!</p>
But now I have switched to the equally awesome Zola</a>!
More on this change in another post, I promise!</p>
p.s. as a sneak pick, head to the project docs</a>,
this should give you an idea of what I have in mind.</p>
Clojure Bites - Mazeboard 5 - More on actions, tests, schema and future plans

Clojure Bites - Mazeboard 4 - From commands to events for a better separation of concerns

One example</h1> Let's have a look at a concrete example, currently the only available action is :move-player</code> so yeah we don't much choice :)</p> The action itself is simple:</p>

Discuss</h1> Reddit</a></li> Linkedin</a></li> Twitter</a></li> </ul> Code</h1> This post include changes made in two different merge requests:</p>

Code</h1> This post include changes made in two different merge requests:</p>

Clojure Bites - Mazeboard 3 - core.async to update the UI layer

Two layers, two state atoms</h1> We want to use two distinct state atoms for:</p> managing and mutating the game state according to requested actions</li>

Clojure Bites - Mazeboard 2 - Using core.async to decouple game and UI logic

Clojure Bites - Mazeboard 1 - Dumdom event handler

Clojure Bites - Mazeboard 0

Checkpoint 2023-11-01

[2023-11-01] What's up?</h1> Hello everyone, long time no see!</p> After a break I've finally found new topics to work on and write about but, before jumping to the upcoming plans lets take a look of what has happened meanwhile.</p>

Clojure bites - Profiling with Tufte

Sending pstats to a log collector</h1> Next sections are about setting Opensearch and mulog as a way to send profiling data "somewhere" else. If you are already familiar with these topics you can skip these details (or maybe read it anyway and help me to fix mistakes ;) ).</p>

Opensearch</h2> To simplify (quite a lot) we can setup a local single node opensearch, available at port 9200, with its nice dashboard, available at port 5601 accessible with admin:admin crendentials, using docker compose and the following docker-compose.yml:</p>

Putting it all together</h1> So far we have seen:</p> How to profile our code</li> How to get pstats in different ways, on demando or accumulating it</li>

Updates</h1> From comments in Reddit I've learnead about alternatives for profiling and tracing</p>

Clojure bites - SSE with Aleph and Reitit

Frontend</h1> In the previous step we have added a way to serve static files, now we can add a basic web page where to render events as we get them. First of all lets see if we can correctly serve a simple HTML file; paste the following content in resources/public/index.html</code></p>

Migrating to Zola SSG

Candidates</h1> Cryogen</a></h2> Language: Clojure Documentation: While complete the feeling is that it does not flow too well</p> Pros:</p> You have the engine at your hand and you can hack on it easily while running</li>

Cryogen</a></h2> Language: Clojure Documentation: While complete the feeling is that it does not flow too well</p> Pros:</p> You have the engine at your hand and you can hack on it easily while running</li>

Nikola</a>/Pelican</a></h2> Language: Python Documentation: Complete</p> Pros:</p> Battle tested</li> Extensible (with lots of extensions)</li>

Hugo</a></h2> Language: Go Documentation: Complete</p> Pros:</p> Battle tested</li> Extensible (with lots of extensions)</li> Lots of themes (I mean LOTS!)</li>

Clojure Bites - Ring basic auth

Skateboarding, keep track of your progress

Overview</h1> Keep track of your skateboarding progress, set your goals and create a plan to achieve them at your own pace.</p>

Copy for landing</h1>

Longer term vision</h1> For amateurs going pro</li> For teams, either brand or national</li> Live coaching, talk to a pro, show videos and get help</li> Workout coach</li> Tools for content creation</li> Public challenges with sponsors</li> </ul>

Just release Unrefined's commit 300, an update!

Just deployed commit 300!</h1> I was working on Unrefined, tested some changes, fixed a bug, cooked a release and suddenly I've realized that it was commit 300! This requires an update post for sure ;)</p>

Closing</h1> My goal with this post is to give an update on Unrefined and what is coming next, it could have been a tweet but it is too long :D</p> See you on commit 400! But most probably even earlier ;)</p>

Clojure digressions - Setting up a playground

Opening wave 2022-06-16

Starting Wave 2023-06-16</h1> After a two days break I am ready to open the new wave, that will last one month like the previous one, I feel comfortable with this pace!</p> What's in the plan? It be broken down to the following items.</p>

Live streaming</h2> Still unsure how to drive more value while streaming, I guess I'll keep trying to learn how to make something out of it.</p>

Closing</h2> This post sets the start of the new wave, the feeling is that the plan is more clear compared to the previous one, I hope that the closing post will confirm that it was a good plan, or at least that I've learned how to improve for the next time, see you around the net!</p>

Closing wave 2022-05-17

Overview</h1> Every wave is a great source of learning material, built on top of the previous experience and the expectations I set for myself. So what have I learned this time?</p> To summarize:</p>

Some metrics</h2>

Projects</h2>

Clojure bites - Rendering HTML

Overview</h1> There are many ways to produce HTML in Clojure, from template engines to data oriented DSLs and React wrappers. Each one has it own use cases, benefits and downsides so there is no one solution that fits all problems.</p>

Placeholders</h3> Placeholders are identified by double curly braces, for example {{blog.tile}}, and essentially take the content of a variable and render it to the output, taking from the blog example:</p>

[2023-11-01] What's up?</h1>
Hello everyone, long time no see!</p>
After a break I've finally found new topics to work on and write about but, before jumping to the upcoming plans lets take a look of what has happened meanwhile.</p>

Sending pstats to a log collector</h1>
Next sections are about setting Opensearch and mulog as a way to send profiling data "somewhere" else. If you are already familiar with these topics you can skip these details (or maybe read it anyway and help me to fix mistakes ;) ).</p>

Updates</h1>
From comments in Reddit I've learnead about alternatives for profiling and tracing</p>

Overview</h1>
Keep track of your skateboarding progress, set your goals and create a plan to achieve them at your own pace.</p>

Longer term vision</h1>

For amateurs going pro</li>
For teams, either brand or national</li>
Live coaching, talk to a pro, show videos and get help</li>
Workout coach</li>
Tools for content creation</li>
Public challenges with sponsors</li> </ul>

Just deployed commit 300!</h1>
I was working on Unrefined, tested some changes, fixed a bug, cooked a release and suddenly I've realized that it was commit 300! This requires an update post for sure ;)</p>

Closing</h1>
My goal with this post is to give an update on Unrefined and what is coming next, it could have been a tweet but it is too long :D</p>
See you on commit 400! But most probably even earlier ;)</p>

Live streaming</h2>
Still unsure how to drive more value while streaming, I guess I'll keep trying to learn how to make something out of it.</p>

Closing</h2>
This post sets the start of the new wave, the feeling is that the plan is more clear compared to the previous one, I hope that the closing post will confirm that it was a good plan, or at least that I've learned how to improve for the next time, see you around the net!</p>
