Machine Learning in Clojure with libpython‑clj: Using Bayesian Networks for Smarter, Interpretable AI [Series 2]

This is the second part of our Machine Learning in Clojure with libpython‑clj series. If you missed Series 1, we covered how Clojure can use Python’s ML libraries through libpython‑clj. In this article, we focus on Bayesian Networks. We will show how to train them, run queries, and use them in Clojure.

Why This Series Matters

Machine learning is not only for Python. With libpython‑clj, Clojure teams can use PyMC, scikit‑learn, and pgmpy. And they can keep the JVM and the functional style they like. The aim is simple: make ML in Clojure clear, practical, and ready for production.

Why Bayesian Networks?

Bayesian Networks are good when you need clarity. They model uncertainty. They use domain knowledge. And they answer “what if?” questions without guesswork.

➤ Small to medium datasets.

➤ Compliance‑heavy work in healthcare, finance, and logistics.

➤ When explainability is required.

This walkthrough builds a simple BN in Python. Then run it from Clojure using libpython‑clj. The process is clear.

Flexiana: Clojure + ML in practice

Flexiana is a Clojure consultancy. We help teams connect Clojure and ML in real projects. We share code, write about patterns, and ship systems that are easy to reason about. If you need support with interop, pipelines, or interpretable models, we are a solid partner.

“Bayesian reasoning helps teams make better calls in logistics, healthcare, and fintech. With Clojure’s REPL and Python’s ML tools, you move faster and stay confident.” – Flexiana ML Team

Why Bayesian Networks Deserve Your Focus

What Are Bayesian Networks?

A Bayesian Network is a directed acyclic graph. Each node is a random variable. Each edge shows how one variable depends on another. The graph encodes conditional probabilities that describe how events influence each other.

Bayesian Networks are not just about guessing what will happen next. They show why something is likely or not, based on how the graph’s parts connect. When you are dealing with uncertainty, these networks give you both a prediction and a peek behind the curtain.

How They Differ From Neural Networks

Why This Distinction Matters

Neural Networks (NNs) really stand out for image recognition, speech, and other messy, unstructured data. They work best with huge datasets and find hidden patterns.

Bayesian Networks fit different needs:

➤ When datasets are smaller, but domain knowledge is strong

➤ When decisions must be explained to stakeholders

➤ When uncertainty needs to be modelled clearly

With all these considered, Neural Networks find hidden patterns. Bayesian Networks help with clarity, reasoning, and trust. 

Why and when Bayesian Networks Beat Neural Networks

❶ Interpretability (Explainable AI)

You can actually see how Bayesian Networks think. Every variable and every link is there in the graph. So if something happens, you can trace the whole path back and explain it. That is a big deal in places like healthcare or finance, where you have to tell regulators or your team exactly why a decision was made. With Neural Networks, they usually keep that logic hidden, which just isn’t good enough when you need transparency.

❷ Explicit Handling of Uncertainty

Bayesian Networks handle uncertainty directly. They do not just return one answer but give you probabilities, so you know what might happen and what’s not, and how sure the model is. That helps when your data is messy or incomplete. Neural Networks, on the other hand, usually just pick one outcome, which can be misleading when things are not clear.  

❸ Encoding Prior Knowledge

Bayesian Networks can include domain knowledge. You can turn relationships and rules from the real world into edges and probability tables. This keeps the model grounded, especially when you do not have a ton of data. Neural Networks, on the other hand, need huge datasets to learn patterns and are not great at encoding expert rules.

❹ Adoption In Practice

PyMC Labs recently pointed out that “Bayesian modelling is really shaping business decisions these days. Probabilistic forecasting models are popping up all over the place- in retail, finance, energy, you name it. Businesses are leaning into these approaches more and more”.

Flexiana Tie‑In

Flexiana focuses on interpretable ML for compliance‑heavy work. Our projects lean on clear reasoning and stable interoperability. Pairing Clojure’s functional style with BN reasoning helps teams build systems that are practical and explainable.

Real‑World Applications of Bayesian Networks

Where BNs Excel

➤ “What if?” analysis: If a shipment is delayed, what is the churn risk? BNs model scenarios and return clear probabilities.

➤ Small or medium datasets: Expert knowledge– Encode known relationships directly. Useful when data is limited.

➤ Compliance‑heavy industries: Interpretable reasoning– Show why an outcome is likely. Fits healthcare, finance, and logistics.

➤ Flexiana case mentioned: Decision support– Flexiana has used BNs for logistics and healthcare clients. The focus is on clear reasoning and compliance‑friendly workflows.

Where BNs fall short

➤ High‑dimensional inputs: Images and audio involve thousands of features. BNs struggle at this scale.

➤ Unstructured data: Text, images, and raw audio need feature extraction. Deep learning handles this better.

➤ Arbitrary function approximation: Neural Networks capture complex, nonlinear patterns. BNs are built for probabilistic reasoning, not every function shape. 

Implementing Bayesian Networks in Practice

Example in Python

Here’s a basic example with pgmpy, a Python library. It builds a small Bayesian Network with two nodes and one dependency.

What this does:

➤ Builds a BN with two variables: Rain and Traffic.

➤ Shows that rain raises the chance of traffic

➤ Checks that the model and probabilities are valid

External Resources

For more details, check the official documentation:

➤ pgmpy Documentation

➤ PyMC Documentation

Translating BN to Clojure with libpython‑clj

Step-by-step interop workflow

➤ Train in Python: Build and check the BN with pgmpy or PyMC.

➤ Load via libpython‑clj:  Import the Python model into Clojure.

➤ Wrap inference in Clojure:  Write small functions for queries and “what if?” checks.

Clojure code snippet

See Series 1 for the basics of ML in Clojure with libpython‑clj.

Flexiana Insight

➤ Enterprise JVM focus: Flexiana’s tutorials show how to bridge Python ML with Clojure in enterprise JVM stacks. The focus is clear interop, stable deployment, and explainable models.

Performance & Optimization Tips

➤ Utility functions for boxed math: Avoid unnecessary boxing. Use primitives where possible and keep Python calls lean.

➤ Batch calls for efficiency: Run queries in groups to cut overhead.

➤ Caching strategies: Cache fixed CPDs and reuse common results. Memoize repeated “what if?” checks.

Bayesian Networks in Action

Case Study ❶: Supply Chain Risk Analysis

Bayesian networks are great for running “what if?” modeling in logistics. If a shipment gets delayed, these models help you see the chances of supplier bottlenecks, missed deliveries, or even losing customers. Managers do not have to guess; they get clear numbers and can actually plan around them.

➤ Use case: You can map out how a delay might lead to customer churn, or spot risks up and down the supply chain.

➤ Output: You can hand these probabilities to your ops or finance teams and actually explain what’s behind them.

➤ Value: You get smarter backup plans and fewer surprises.

Case Study ❷: Healthcare Diagnostics

BNs connect symptoms, test results, and conditions, showing how each piece of information changes the probabilities of a diagnosis. The reasoning is not hidden- clinicians can see not just what the model predicts, but how sure it is.

➤ Use case: If you plug in a set of symptoms, you get a clear picture of which conditions are most likely. 

➤ Output: The logic stays out in the open, so anyone reviewing the case can follow every step. 

➤ Value: This kind of transparency is good when you need to explain decisions or meet strict compliance rules.

Employing Bayesian Networks for the Diagnosis and Prognosis of Diseases: A Comprehensive Review (arXiv, 2023) – This paper gives an overview of how Bayesian Networks support medical diagnosis and prognosis. It shows how they use medical knowledge and manage uncertainty in decisions.

Case Study ❸: Financial Fraud Detection

BNs do not just spot fraud- they break down what made a transaction suspicious in the first place. You get to see exactly which factors raised the red flag, not just a vague alert. That kind of clarity makes audits and regulator checks way smoother.

➤ Use case: They scan transaction patterns and highlight the risky ones.

➤ Output: Real reasons for every alert, not just a score.

➤ Value: You end up with detection you can actually trust and clearer investigations.

According to a 2025 industry survey, more than 70% of financial firms have adopted probabilistic or machine learning models to detect fraud, reflecting a clear shift away from rule‑based systems.

Why This Matters for Your Team

Business Case for Clojure + ML

➤ Faster orchestration and deployment: For starters, you can move fast. The REPL lets you test ideas and make changes quickly, so updates are faster without waiting around for long builds.

➤ Seamless JVM integration: Clojure integrates nicely with the JVM. You can plug ML models right into your current systems. No need for extra layers or awkward workarounds.

➤ Lower barrier for Clojure teams:  if your team already knows Clojure, you do not have to rebuild your stack or retrain everyone just to bring machine learning into the picture. You get to use the tools you know, and still take advantage of ML.

Explore Flexiana’s consulting to fit ML into your Clojure stack. Interop, deployment, and compliance‑friendly workflows: that’s the focus.

Python and Clojure Examples for a Simple Bayesian Network

Python: Define a small BN, add CPDs, and run inference

➤ Structure: A clear DAG with three variables; Cancer has two parents.

➤ Nodes: Random variables with named states.

➤ Edges: Smoking and Pollution feed into Cancer.

➤ CPDs: Tables capture prior knowledge and uncertainty.

➤ Inference: Use variable elimination for “what if?” queries.

Clojure: Build and query the same BN via libpython-clj

This example uses libpython‑clj to import pgmpy, set up the network, and run queries in Clojure.

Interop flow:

➤ Build: Mirror the Python BN and CPDs with libpython‑clj.

➤ Infer: Call pgmpy’s VariableElimination from Clojure.

➤ Return: Get Python objects, then print or convert to Clojure data.

Wrap Python objects into idiomatic Clojure maps

Label conversion: Map states to values, e.g., {“no” 0.90, “yes” 0.10}.

Engagement tip: Log queries and results to show stakeholders how each node shapes outcomes.

FAQs (People Also Ask)

❶ Are Bayesian Networks the same as Neural Networks?

Not at all. Neural Networks learn patterns from tons of data, while Bayesian Networks focus on cause and effect, mapping out probabilities and showing you how they conclude.

❷ Can Bayesian Networks handle big data?

Honestly, they are not built for that. Bayesian Networks help with small or medium datasets, especially when you need expert input. If you have massive, unstructured data, deep learning usually does a better job.

❸ Why use Clojure for ML instead of Python?

Clojure runs on the JVM, so it fits right into enterprise setups. Plus, if you need something from Python, you can just call those libraries- no need to pick one or the other.

❹ How does libpython-clj simplify interop?

It makes things easy. Clojure can import Python ML libraries directly- train your model in Python, then query it from Clojure. No need to build complicated bridges between the two.

❺ What industries benefit most from Bayesian reasoning?

Healthcare, finance, and logistics rely on Bayesian models. They want explainable results, clear probabilities, and the ability to test out “what if?” scenarios quickly.

Conclusion

Machine learning in Clojure is practical for real teams. With libpython‑clj, you can use Python’s ML libraries while staying in the JVM stack you already trust. That means faster iteration, smoother deployment, and less friction for Clojure developers.

Bayesian Networks add clear value. They do not just predict; they show the reasoning. This matters in healthcare, finance, and logistics, where decisions carry weight. BNs handle uncertainty and map cause‑and‑effect, so managers and auditors can see why a result makes sense.

If your team is exploring ML in Clojure, now is a good time to try it. Share your thoughts, compare notes, and check the sample code in our GitHub repo. 

If you want help, Flexiana’s consulting team can guide design, deployment, and integration so it fits your stack.

The post Machine Learning in Clojure with libpython‑clj: Using Bayesian Networks for Smarter, Interpretable AI [Series 2] appeared first on Flexiana.

Permalink

map operations

Code

;; map_ops.clj

(defn maps-op ([func map-1 map-2 default-value]
               (let [map-1-keys (keys map-1)
                     map-2-keys (keys map-2)
                     keys (distinct (concat map-1-keys map-2-keys))]
                 (->> (map #(assoc {} %
                                   (func
                                    (get map-1 % default-value)
                                    (get map-2 % default-value))) keys)
                      (apply merge))))
  ([func map-1 map-2]
     (maps-op func map-1 map-2 nil)))

(defn map-op
  ([func map-1]
   (let [keys (keys map-1)]
     (->> (map #(assoc {} %
                       (func
                        (get map-1 %))) keys)
          (apply merge))))
  ([func m val]
   (maps-op func m {} val)))

(maps-op + {:a 1 :b 2 :c 5} {:a 3 :b 4} 0)
;;=> {:a 4, :b 6, :c 5}

(maps-op + {:a 1 :b 2} {:a 3 :b 4})
;;=> {:a 4, :b 6}

(map-op + {:a 1 :b 2 :c 5} 5)
;;=> {:a 6, :b 7, :c 10}

(map-op inc {:a 1 :b 2 :c 5})
;;=> {:a 2, :b 3, :c 6}

;; how it works 

(def map-1 {:a 1 :b 2})
(def map-2 {:a 3 :b 4})

(def map-1-keys (keys map-1))

map-1-keys
;;=> (:a :b)

(def map-2-keys (keys map-2))

map-2-keys
;;=> (:a :b)

(distinct (concat map-1-keys map-2-keys))
;;=> (:a :b)

(+ (get map-1 :a) (get map-2 :a))
;;=> 4

(assoc {} :a
       (+ (get map-1 :a) (get map-2 :a)))

;;=> {:a 4}
                         
(def func +)

(map #(assoc {} %
              (func
               (get map-1 %)
               (get map-2 %))) '(:a :b))
;;=> ({:a 4} {:b 6})

(merge {:a 4} {:b 6})
;;=> {:a 4, :b 6}

(apply merge [{:a 4} {:b 6}])
;;=> {:a 4, :b 6}

Permalink

Edge-Aware Pixelation for Better Pixel Art

Pixelation is everywhere these days, from retro game aesthetics to modern design trends. But if you've ever tried to create pixel art from a photograph, you've probably noticed that the results often look a bit off. Edges get jagged, important features get distorted, and the whole image tends to lose its character.

Most pixelation algorithms rely on the same approach of downscaling the image, then upscaling it back. While this is fast and cheap to implement, it treats every part of the image the same way forcing a rigid grid that cuts indiscriminately across edges, faces, and fine details.

In this post, we'll take a look at a strategy of using edge-aware pixelation. The idea here is to use an algorithm that adapts to the structure of the image rather than forcing a uniform grid onto it. We'll look at how it works, why it tends to produce better results, and the trade-offs involved.

The Traditional Approach

Let's start by looking at what most pixelation libraries do under the hood. The most common approach uses standard image scaling with smoothing.

// Pseudo-code for traditional pixelation
const downscaled = scaleDown(image, pixelSize); // Uses bilinear/bicubic
const pixelated = scaleUp(downscaled, originalSize); // Nearest-neighbor

While this works, there are a few obvious problems. Smoothing filters tend to blur important edges before downsampling, and the pixel grid doesn't care about the image content. This often leads to artifacts where edges get "chopped" across pixel boundaries, creating jagged lines. You also end up losing fine features like eyes or text.

Some approaches try to improve on this using median color sampling instead of averaging.

// Sample colors in each block, pick median
const blockColor = median(colorsInBlock);

This avoids some of the blurring, but it still suffers from the same fixed grid issue. It ignores the image structure and can create harsh transitions between blocks.

Basically, all these methods force a uniform grid onto the image without considering its content. The result is usually pixel blocks that cut across important features. For example, if we use the following image from wikimedia as the input.

The result ends up looking something like the following when using a naive pixelation algorithm:

Edge-Aware Adaptive Grid

Instead of forcing a rigid grid, we can let the grid adapt to the image. This is the core idea behind edge-aware pixelation. We can treat pixelation as an optimization problem with four main stages.

1. Edge Detection

First, we need to understand where the important features are in the image.

// Sobel operators detect gradient magnitude and direction
const gradient = calculateGradient(image);
const edges = applyNonMaximumSuppression(gradient);
const edgeMap = thresholdEdges(edges, sharpness);

We use Sobel operators to compute the gradient magnitude, and then apply non-maximum suppression to thin the edges to a single-pixel width. Finally, we use percentile-based thresholding to adapt to the edge distribution of the specific image. Since this can be computationally expensive, using WebGL can provide a significant speedup here.

This gives us an edge map where bright pixels represent edges and dark pixels represent smooth regions.

2. Grid Initialization

Next, we start with a regular grid matching the target pixel size.

const grid = createUniformGrid(width, height, pixelSize);

Unlike traditional methods, the grid is just a starting point. Each cell is defined by four corner points which we can move around.

3. Grid Optimization

This is where the actual work happens. We iteratively move the grid corners to align the cell boundaries with the detected edges.

for (let iteration = 0; iteration < numIterations; iteration++) {
  for (each corner in grid) {
    // Search nearby positions
    const candidates = searchNeighborhood(corner, stepSize);

    // Evaluate how well each position aligns edges
    for (const candidate of candidates) {
      const score = evaluateEdgeAlignment(candidate, edgeMap);
      if (score > bestScore) {
        bestPosition = candidate;
      }
    }

    // Move corner toward best position (with damping)
    corner.moveToward(bestPosition, damping);
  }
}

For each corner, we test multiple positions in a local neighborhood and evaluate the alignment by sampling edge strength along the grid edges. We want to find positions where the grid boundaries follow the edges continuously. We also use damping to prevent over-optimization and maintain stability.

The result is a grid that bends and adapts to align with the natural structure of the image.

4. Color Assignment

Finally, we need to assign colors to our optimized cells.

for (each cell in grid) {
  const pixels = samplePixelsInCell(cell);

  // Blend between average (soft) and median (crisp) based on edge presence
  if (cell.hasEdges) {
    color = blend(average(pixels), median(pixels), sharpness);
  } else {
    color = average(pixels); // Smooth regions use average
  }

  renderCell(cell, color);
}

Here we can use a blending strategy. For smooth regions, we use the average color for natural blending. For edge regions, we can blend between average and median based on the desired sharpness. This lets us tune the aesthetic from soft, blended edges to crisp, high-contrast ones. Looking at the two methods side by side, we can see how much smoother the resulting image is:

Edge detection	Naive

Discussion

There are several advantages to this approach. The most obvious is edge preservation. Traditional methods create jagged artifacts because pixel boundaries cut across edges. By aligning the grid boundaries with the edges, we can preserve continuity and create smoother transitions.

This also means we don't get those choppy edges on outlines, and we can preserve fine details like facial features or text much better. The algorithm effectively has some semantic awareness of what's important in the image.

However, there are trade-offs to consider. The adaptive grid and color blending can produce softer edges compared to traditional methods. If you're looking for extremely crisp, high-contrast pixel art with hard edges, like what you'd see in retro games, traditional downsampling might actually be a better fit.

You also get less contrast in some cases. The color blending can reduce the overall "punchiness" compared to median sampling.

Performance is another factor. Edge-aware pixelation is computationally more intensive. You have to handle edge detection, iterative grid optimization, and spatial hashing for rendering. While WebGL optimizations make it practical taking 100-500ms on most images, simple downsampling will always be faster.

I've found that edge-aware pixelation works best for photographs, portraits, and complex scenes where preserving structure is important. It's less ideal for abstract art or images where a uniform grid is desired for stylistic reasons.

Conclusion

By detecting edges and optimizing alignment, edge-aware pixelation produces pixel art that does a good job of preserving the essence of the original while still achieving low-resolution aesthetic.

If you're interested in trying this out live here, and all the code is available as an open source library called Pixel Mosaic that implements both traditional and edge-aware pixelation.

I hope this gives you some ideas for your own image processing projects. Sometimes the simple method is enough, but for complex images, the extra effort of edge-aware processing can be well worth it.

Permalink

My most useful tools of 2025

Besides updating the About page, I wanted to start writing again about the various tools and solutions I’m using more frequently. Lately, the tools are more in the physical realm than in software. (But I will note here, that 2025 is finally the year that I switched to neovim for myself instead of vim. More on this at the end of the list.)

Slice 10512 Pen Cutter

I’ve tried carrying EDC pocket knives for opening boxes, and I’ve tried stocking cheap scissors in drawers all over the house. In the end, the best way to open boxes turns out to be these ceramic-bladed box openers from Slice. The blades aren’t sharp enough to cut a finger, but they will open tape, including that tricky tape that has threads in it. When you let go, the blade retracts, and can’t poke you when you reach into a pocket or a drawer. It sounds basic, but given how much I get shipped to me for convenience, this has been one of my top tools for the last few years.

Magnetic parts bowls

I try to fix things myself when I can. If you DIY stuff, you’ve probably run into this problem: screws like to roll off surfaces and disappear. I got tired of this, and since most screws are made from ferrous metal, these little magnetic parts bowls solve the problem for me. I’ve got a few upstairs near where I practice guitar and where my home office desk is, and a few more downstairs on my work bench. Conveniently, the magnet in the base will also make the bowl stick to anything iron or steel, too, so I tend to be able to “plunk” down a bowl on something like a desk frame or my work bench vise, and know that all the screws for a project will stay in there. (Provided they’re ferrous.) Compared to random jars or recycled plastic dishes full of screws, this feels like a big upgrade for me, and so that is why this item is second on my list for 2025. There’s lots of variations of these available online or at your local stores – I recommend trying to find them in the local hardware store.

Wowstick electric screwdriver

Even with several complete small screwdriver sets and an electric drill that can fit screwdriver bits, I wanted something small that I could keep in my office and use for screwdriver tasks. I imagined it could help with taking apart keyboards or computers, and maybe occasionally for DIY projects around the house. About a year ago, I’d read several reviews of these Wowstick electric screwdrivers online and decided to buy one. I’m not sure exactly what model mine is, but it is larger than a pen and much smaller than most full-sized (analog) screwdriver handles that I have. It came with a large number of screw bit tips, which are unfortunately a smaller hex head size than the typical screwdriver tips. But along with my ifixit magnetic screwdriver set, I’ve got just about any type of screw covered. The Wowstick screw driver charges with Type-C USB, and due to the planetary gear box, it seems fine to just use it like an analog screwdriver by turning the whole screwdriver body, to get stubborn screws unstuck. It also helps me to not over-torque screws on things like small custom keyboards and prevents stripping out heads on tiny laptop screws.

Field Notes notebooks

I’ve collected notebooks for a long time, but most of the journal-sized notebooks are too large to carry around all the time. The typical Field Notes softcover notebook fits in a jeans pocket with a pen. Because I can always have one with me, these have become handy for me to use for all sorts of quick notes, daily TODO lists, for drawing out diagrams and jotting down measurements as I’m working on something. There’s all kinds of special editions of these, but I generally tend to use either basic dot grid notebooks or their TODO list variant. I can recommend these highly for the cost over other little pocket notebooks – they’re just durable enough that after using them daily for a full notebook’s worth, the covers and spines are just broken in, but they don’t start falling apart like cheaper notebooks or spiral-bound pocket notebooks.

Neovim

The only software on this list, and the only free thing. Neovim is a modern vim. I’ve been primarily a vim user for the last 10 years (with side quests into emacs for Clojure, Lisp/Scheme and then to use org-mode). I made the switch to Neovim recently because I’d seen how powerful Language Server Protocol (LSP) was for Rust development in VS Code, but I wanted it in my usual vim editor. LSP will show you problems beyond simple syntax errors, right in your editor buffer, including compile and type errors, incorrect function names, and names of functions that are correct but aren’t imported to the current scope. I’m running a very minimal setup that still gets me LSP support for most things out of the box, and it has been great. I can highly recommend the switch if you’re already familiar with vim.

Conclusion

That’s a wrap on 2025. Happy new year, everyone.

Permalink

Image Processing with dtype-next Tensors

Permalink

Clojure Deref (Dec 10, 2025)

Permalink

Clojure 1.12.4

Permalink

GitHub Copilot coding agent with Clojure

Copilot coding agent is a service provided by GitHub that supervises Large Language Models within Actions runners to complete prompted coding tasks. Users can interact with the LLM using standard GitHub features like issue assignments, comments and pull request reviews, but also handy buttons littered across the UI (look for the cloud-surrounded >_ icon).

While these familiar user controls make coding agents immediately accessible, what really sets this apart from other approaches is security. Agents are sandboxed and firewalled within their runners, with write access only to their copilot/* branch. Even once they push, CI triggers still require explicit approval from an authorized user. Furthermore, repo secrets are sequestered from the coding agent, with only explicitly enabled secrets accessible to the coding agent (via the copilot environment at github.com/<org>/<repo>/settings/environments). GitHub have done a great job applying the fork+pr security model for coding agents—maintainers can treat coding agents as any other untrusted collaborator.

To get started, first check your Copilot plan at github.com/settings/copilot. You may have complimentary access as an open source developer (I did). Copilot coding agents use “premium requests”, as of writing one per prompt. Here you can see your quota, if any.

Go to github.com/copilot/agents to write a prompt. Pick a Clojure repo and write a detailed prompt. Easier said than done, so let’s walk through an example.

In previous explorations, I found LLM’s did not know how to use clojure.java.process in Clojure scripts, so I created a repo of examples for training data in frenchy64/clojure.java.process-examples. The problem was, I didn’t know how to use it either, so I was hoping a coding agent could figure it out by itself. Here’s my initial prompt:

This repository is meant for supplementary documentation and tests for the new Clojure API clojure.java.process. Part of the problem is that the documentation is sparse and AI agents have very little example code to train on. So this task will be challenging to complete for an AI agent: I want you to fill out the test suite covering examples of a wide range of use-cases, especially common, for clojure.java.process. I have set the repo up to help you. You must install Java and Clojure. Then you should use clojure -M:test to run the tests. It uses cognitect’s test-runner so filter the tests as needed. Please run the test suite as you work. Please cover examples of every function in clojure.java.process, and every option it supports. As well as common use-cases, I’m especially interested in ones that coordinate concurrent processes. For example, a process that is running a Clojure REPL accepting input from and piping out to from another process that is reading from stdin (make sure you use clojure and not clj to avoid issues with readline). Look at these places for inspiration: Java documentation for Process, babashka’s process library. Add at least 100-150 tests. The tests should pass relatively quickly, aim for 10-20 seconds. Faster is better. Finally, enhance the README.md with all the examples you use, ensuring they match exactly to the example in the test suite (but strip away the testing scaffolding if it’s clearer to use in the documentation).

I find higher success when giving coding agents ways to check their own work, in this case running tests. This prompt was only the beginning though—you can find subsequent prompts in Copilot’s pull request. The PR evolved into machine-checked documentation suitable for both human consumption and training data. Notice you should mention @copilot only once per prompt to get its attention, especially relevant for requesting changes over multiple files.

You should set up Copilot’s environment via .github/workflows/copilot-setup-steps.yml to save having to include it in the prompt and help the LLM get right on task. The setup steps execute just before the firewalled coding agent gains control. Typed Clojure’s Copilot setup steps is a good starting point. It installs several distributions of Clojure and downloads deps from clojars. fully-satisfies’ Copilot setup steps additionally shows how to set up Leiningen. With this setup, you can keep firewall exceptions minimal. If an agent coding session requires a new clojars dep, it should add them to your deps and the next session will download them.

Here are some tips. Setup steps must be on the default branch. To use on forks of open source projects, change the default branch to one that includes the setup steps. The timeout-minutes of the setup steps dictates coding agent’s session duration. I found 30 minutes to be a good starting point, modifying your prompts to direct the agent to break up tasks into chunks if needed. Rate limits are problematic, especially on non-Enterprise plans. In practice, I found 2-3 concurrent coding agents stay within these limits, bumping up to 4-5 on Enterprise plans.

Permalink

Gemma 3 AI model in Clojure

Recently I've been working on the ONNX runtime integration into Deep Diamond, backed by the grant sponsored by the Clojurists Together Foundation. In the past few articles, we've seen how ONNX models are integrated into Deep Diamond, using only a single function onnx, with almost no need for additional configuration (which is available). I used a simple MNIST model in the demonstration. But, can we now load and run the inference on the real deal models, such as the open LLMs from the Hugging Face, for example? Let's see!

The Hugging Face model card has this to say about Gemma 3: "Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models." (etc., etc.) So, it seems to be something worth trying.

I'll try to be brief, and skip the unnecessary talk. Let's just show the code, which I've just lifted up and adapted from the Diamond's midje tests.

What we need for this? First, decide on the backend engine; this time we'll use tensors in main memory backed up by the oneDNN engine (DNNL).

(def fact (dnnl-factory))
(def neand-fact (neanderthal-factory fact))

Next, load and configure a particular flavor of Gemma 3 (a smaller one, only 1 billion parameters). The onnx function creates a generalized blueprint, which can create the actual functions when evaluated with the specific input tensors.

(def onnx-bp (onnx fact "data/gemma-3-1b-it-ONNX-GQA/onnx/model.onnx"
                   {:options (-> (options)
                           (override-dimension! "batch_size" 1)
                           (override-dimension! "sequence_length" 1)
                           (override-dimension! "past_sequence_length" 1)
                           (override-dimension! "total_sequence_length" 1))})

Gemma 3 has 63 inputs and 61 outputs. We'll need to provide these, but even here we can automate some parts with Clojure, since past-key values are pretty uniform. We only need to provide inputs, while the engine can create the outputs for us.

(def src-tz (tensor fact [1 1 28 28] :float :nchw))
(def input-ids (tensor neand-fact [1 1] :long :nc))
(def position-ids (tensor neand-fact [1 1] :long :nc))
(def attention-mask (tensor neand-fact [1 1] :long :nc))
(def past-key-values (repeatedly 60 #(tensor fact [1 3 1 64] :float :nchw)))

Next, create the executable instance model. Nothing too fancy here.

(def gemma-next! (onnx-bp (into [input-ids attention-mask position-ids] past-key-values)))

Now, these inputs need to be initialized. Normally, that would be done inside an LLM generation loop, but here we only demonstrate one step, and we transfer some mock data.

(transfer! [2] input-ids)
(transfer! [0] position-ids)
(transfer! [1] attention-mask)
(doseq [pkv past-key-values]
  (transfer! (repeat 0) pkv))

Aaaaand, we actually run the model by calling our gemma function, which provides the next token.

(gemma-next!)

Now, hold on with the celebration. This does not actually return a full answer from the LLM. This only returns the next token, but in the form of large tensor full of numbers. The information is there, but needs to be extracted from these numbers to the form of string. Also, this is only one step; a LLM would typically run this in a loop and spew tokens after tokens. There's some more work to do until we get a ready made, hands-off chatty LLM. But the main work has been done, and now it's the matter of setting it up properly, tokenizing the inputs, and calling it in a useful way! Still lots of work, but not the hardest parts :)

I've applied for Clojurists Together yearly funding in 2026. If you are a Clojurists Together member, and would like to see continued development in this area, your vote can help me keep working on this :)

My goal with this funding in 2026 is to continuously develop Clojure AI, ML, and high-performance ecosystem of Uncomplicate libraries (Neanderhal and many more), on Nvidia GPUs, Apple Silicon, and traditional PC. In this year, I will also focus on writing tutorals on my blog and creating websites for the projects involved, which is something that I wanted for years, but didn't have time to do because I spent all time on programming.

Permalink

What are formal systems?

I work at Metabase, and I like my job. If you want to work at Metabase and like your job, come apply to work here! I program in Clojure (though the frontend folks work in TypeScript), everyone is smart, and it’s 100% remote with people all over the world. The code is open source, and it’s used by UNESCO. Apply now.

What are formal systems?

When I’m drafting my book, I often get a glimpse of the underlying structure the ideas I’m presenting are built on. These ideas seem so important. But alas, they’re often very abstract and not directly relevant to the skills I’m trying to teach to a particular audience. This newsletter and my podcast are a way to write/talk about those ideas to a more sympathetic audience.

My book is about domain modeling. One ideal to strive for when building a domain model in code is to construct it as a formal system. When I first got that idea, I do what I typically do, which is to start writing about it. But soon I realized that I didn’t really know what formal meant in that term. I did some research. And it’s a really important idea, so I’m going to share my understanding with you.

Before my research, if you asked me what formal system meant, I would tell you that it’s about being rigorously defined, as in “let’s get a formal definition.” But that’s not it, really. Yes, formal system are rigorously defined, but they’re a lot more like programming languages than just “good definition.” Let’s look at an example:

In predicate logic, a kind of formal logic, we can assign variables to a few concepts:

• Jim - the protagonist of this story

• Pizza - the plot device

• Likes - the character development

• Eats - the action

Then we can talk about Jim, like this:

This means that if Jim likes a food, he eats it. Now what happens if we also throw this into the mix?

Jim likes pizza. We can then imply that:

Jim eats pizza. Surprise ending.

Well, okay, it’s not a very interesting story. But it does give us everything we need to define formal system, the real plot of the story. Mathematicians will define it like this:

A formal system consists of:

• symbols (valid characters/tokens) — Jim, Likes, (, ⇒, …

• statements (valid arrangements of symbols) — Likes(Jim, Pizza)

• mechanistic rules (for deriving new statements from existing statements) — the rule that lets us derive our ending

• axioms (statements that are true implicitly)

I think of it in programming terms. The first two items (symbols and statements) are the syntax. The last two (mechanistic rules and axioms) define the semantics.

There’s something about formal systems that is powerful and paradoxical. There’s a strange combination of meaning, meaninglessness, and machine-like coldness that somehow makes them more powerful. And that’s why I find them fascinating, just like programming itself. It’s just electrons moving through a circuit, yet somehow it does meaningful work. Where does the meaning come from? (Hint: It’s us.)

First, the syntax defines a structure, like an Abstract Syntax Tree. An important part of the syntax is that there are symbols that are neutral. These are the variables. We defined variables like Jim and Pizza, but as far as logic is concerned, they could be A and B. Logic doesn’t care about Jim or pizza. Likewise, your compiler doesn’t care what you name your variables, nor does the chip it runs on. And this is important. The variables are only linked to human meaning through their names and other connections to real-world things.

However, some parts of the syntax are meaningful to the system. The structure itself is meaningful (as in F(A, B) indicates a predicate called F with arguments A and B). And some of the symbols have meaning (⇒ means implication). The meaning comes from their use in the mechanistic rules, somewhat how a syntax-driven interpreter would work. The structure of the statements is pattern-matched in the derivation rules, which leads to new statements. The term formal refers to the importance of the structure in defining meaning.

I find this mix of meaningless symbols and meaningful symbols to be a beautiful puzzle. Yes, there is a certain elegance to formal systems like propositional logic. A handful of meaningful symbols (logical connectives) and a few inference rules can ensure you only derive valid arguments. But there’s something more than just elegance at work. There’s something to their limited meaning that makes them practical. Logic wouldn’t work if you tried to include all humanly felt meaning, so you only include a tiny subset that you trust to work. Yet it can find new, if simplistic, results.

One interesting exercise is to find the minimal set of meaningful symbols that can express all the others. In propositional logic, this is the NAND connective (NOR works, too). The SK combinator calculus is Turing complete. And you can derive both S and K from the iota combinator! So iota, just one symbol (and using the lambda calculus rule of application) is Turing complete.

Similarly, Lisp is very minimal (at least the original Lisp). It is defined with three kinds of symbols:

1. Atoms (alphanumeric sequences)

2. (

3. )

The syntax also requires spaces between atoms to distinguish them. The syntax defines two kinds of values:

1. Atoms

2. Lists

Super minimal! The inference rules are also simple:

1. An atom’s meaning is either a number (if it’s all digits) or it’s a symbol, in which case its value is looked up.

2. A list’s meaning is determined by the first value in it.

   1. There are certain symbols with special meanings, like lambda or cond.

   2. Other symbols are looked up and applied like functions to the arguments.

This is such a simple formal system, yet it builds up an entire programming language. I’m leaving out that you probably have to define a lot of the functions yourself in some other language (like machine code), but even that number is small. In other words, Lisps are bootstrapped. Lisps today are more complicated, but they still maintain some taste of the simplicity of the original.

What I think is cool is that programming languages are essentially formal systems. They have all the ingredients. Their semantics are defined by the compiler, which defines them as a translation into another formal system, the machine code. Strangely, machine code does not have any non-meaningful parts. Anything you can express in machine code is meaningful to the semantics of the chip. The only hint of a meaningless thing is that it operates on numbers that themselves are meaningless to it, except in as much as it knows how to do arithmetic on them and fetch memory at a numeric address. They are abstract quantities that represent values meaningful to us. So we can say 1 means Jim and 2 means pizza.

I feel compelled to mention the book Gödel, Escher, Bach right now, for some reason. I have read only a few pages of it, but it’s probably a more playful and complete exploration of the strangeness of formal systems than I can possibly do. Just wait until you see what self-reference in a formal system does to it!

The challenge with using formal systems is in modeling a real-world scenario. It’s an art to translate Jim and his tastes into logical statements such that you can derive new meaningful statements from it. The derivation, once the statements are written, is quite trivial. This challenge of encoding the meaning is one reason why proofs are hard even when something seems obvious. By the way, the art of encoding a set of real-world scenario into a program is known as domain modeling. What are the meaningful parts you want to capture from the domain, and what parts are you leaving up to the humans for meaning?

There’s also a cool balance between minimalism of the system and human habitability. Sure, it’s very elegant that NAND is complete. But just look at the statements you have to write to do anything! They’re unreadable. It’s very hard, as a human, to wrap your intellect around them. Structure dominates. Perhaps you can eventually learn to read it, with time, but you’re mostly just counting parentheses. Check out this expression using the iota combinator:

So clear! (not)

There’s a tradeoff between number of symbols and length/structure of statements. We want to select a set of symbols we can easily juggle in our minds (not too big) that are, at least metaphorically, meaningful to the concrete world we live in. Making the story about Jim and pizza is something we can relate to with almost no effort. Jim eats foods he likes, he likes pizza, so he eats pizza. It’s obvious (I believe) because it taps into the social parts of our brains. We want symbols like ⇒, ∧, and ∨. We can understand them. Software design is often about this tradeoff: Do we inline the code or do we extract a function and give it a meaningful name? Do we want longer code or more symbols?

The last cool thing I want to mention is that this is the key to stratified design and metalinguistic abstraction. In stratified design, each layer defines new meaningful symbols for the layer atop it to use (relying on the underlying lambda calculus rules of inference). But in metalinguistic abstraction, you define an entirely new formal system, with new rules of inference, that hopefully lets you express your problem more easily than lambda calculus. Think a rules engine or SAT solver.

Well, it’s time to conclude. Formal systems are a neat mix of meaning and meaningless parts. The meaningless parts are like holes where we can fill in our own, human meaning. We want to find a set of meanings we can use to derive the answers to our questions. That is not easy. And finding the balance between the number of meaningful symbols and the structure is an art. These are some of the underlying questions we face every day when we program. The questions never go away, but the practice of domain modeling points you to a reliable source of answers: The domain itself.

Permalink

1.12.134 Release

Permalink

Demystifying Scala 3 Macros: A Deep Dive with Quoting and Splicing

Welcome to the world of compile-time metaprogramming in Scala 3! Macros can seem like magic, but they are built on a powerful and elegant system of Quoting and Splicing. Today, we're going to dissect a practical and elegant macro that converts any case class into a Map[String, Any]. By the end of this article, you'll understand not just what this macro does, but how it works, right down to the low-level reflection concepts that power it.

Here is the code we will be exploring:

package com.dev24.macros

import scala.quoted.*

object ToMapMacro {
  // The user-facing, inline method
  extension [A](inline a: A) inline def toMap: Map[String, Any] = ${ toMapImpl('a) }

  // The implementation that runs at compile time
  def toMapImpl[A: Type](instance: Expr[A])(using Quotes): Expr[Map[String, Any]] = {
    import quotes.reflect.*

    // 1. Get the type and symbol of A
    val tpe = TypeRepr.of[A]
    val sym = tpe.typeSymbol

    // 2. Get the list of case class fields
    val fields: List[Symbol] =
      if (sym.isClassDef) sym.caseFields.toList
      else Nil

    // 3. For each field, generate a quoted (String, Any) tuple
    val mapEntries: List[Expr[(String, Any)]] = fields.map { field =>
      val fieldNameExpr = Expr(field.name)
      val fieldAccess = Select(instance.asTerm, field)
      val fieldValueExpr = fieldAccess.asExprOf[Any]
      '{ ($fieldNameExpr, $fieldValueExpr) }
    }

    // 4. Assemble the final Map from the list of tuples
    '{ Map(${Varargs(mapEntries)}*) }
  }
}

Let's break this down piece by piece.

Part 1: The Gateway to Metaprogramming - The inline Method

extension [A](inline a: A) inline def toMap: Map[String, Any] 
= ${ toMapImpl('a) }

This is the method the user will call. It's designed to look and feel like a regular method, but it's anything but.

• extension [A]: This is just modern Scala syntax to add a method to any type A. It's syntactic sugar and makes the call site clean: myCaseClass.toMap.
• inline a: A: The inline keyword is the trigger. It tells the Scala compiler, "Do not execute this method at runtime. Instead, when you see a call to this method, replace it with the result of the code on the right-hand side during compilation."
• ${ ... }: This is the Splice operator. It's the bridge from the "normal" code world to the "macro" world. The code inside the splice is executed by the compiler as it builds your program.

• toMapImpl('a): This is the call to our macro implementation.

  • toMapImpl: The function that will generate the code.
  • 'a: This is the Quote operator (an apostrophe). It takes the value a and lifts it into the compiler's world as a representation of the code that produced a. This is not the runtime value of a; it is an Abstract Syntax Tree (AST) node representing the expression a. We call this a quoted expression. Its type is Expr[A]

In short: When the compiler sees person.toMap, it pauses compilation, calls toMapImpl with a quoted representation of the person expression, gets back a new piece of code (a quoted Map), and splices that new code directly into the program where person.toMap used to be.

Part 2: The Engine Room **- **The Macro Implementation

def toMapImpl[A: Type](instance: Expr[A])(using Quotes)
: Expr[Map[String, Any]]

This is where the real work happens. This function runs inside the compiler during the compilation phase.

• A: Type: This is a Context Bound. It's a request for a Type[A] instance to be available. Type[A] is a "type tag." It's the quoted representation of the type A itself (e.g., Person, not an instance of Person). This is how the macro knows what class it's working with.
• instance: Expr[A]: This is the quoted expression we passed in with 'a. It's the AST for the value being converted (e.g., the AST for Person("Alice", 30)).
• (using Quotes): This is a Context Parameter. Quotes is the macro's toolbox. It provides access to all the reflection and metaprogramming utilities. It's the entry point to the compiler's internal APIs.
• : Expr[Map[String, Any]]: This is the return type. Crucially, the macro does not return a Map. It returns a quoted expression that, when compiled and run, will produce a Map. This is the fundamental contract of a macro: generate code, don't compute runtime values.

Low-Level Reflection: import quotes.reflect.*

To do the heavy lifting of inspecting the class structure, we need to go deeper than the high-level Quotes API provides. import quotes.reflect.* gives us access to the compiler's core reflection types.

• TypeRepr: A rich, internal representation of a type. It's more detailed than Type.
• Symbol: The most fundamental concept. A Symbol is a unique name for any declaration in Scala: a class, a trait, an object, a method, a field, a type parameter, etc. Think of it as the ultimate "key" in the compiler's symbol table.
• Term: A general representation of an expression or statement in the AST. An Expr is a type-safe wrapper around a Term.

Step 1: Inspecting the Type

val tpe = TypeRepr.of[A]
val sym = tpe.typeSymbol

• TypeRepr.of[A]: We use our Type[A] evidence to get the full, rich TypeRepr for A. Represents a type (e.g., User, String, List[Int]). It's the "what is its structure?"

This line is about getting a handle on the type itself.

Line 1: val tpe = TypeRepr.of[A]

• [A]: This is a type parameter. The code must be inside a context where A is known, such as a generic method (def myMethod[A]: ...) or a generic class (class MyClass[A]).
• TypeRepr: This is the central class in Scala 3's reflection API for representing types. A TypeRepr is the compiler's internal, rich description of a type. It can tell you everything about the type's structure.
• .of[...]: This is a summoning method. It asks the compiler: "Give me your internal TypeRepr object for the type I specify here." This operation happens entirely at compile time.

What tpe contains:
If A is List[String], tpe is a TypeRepr that knows:

• Its base type is scala.collection.immutable.List.
• It has one type argument, which is scala.Predef.String.
• It's a concrete, applied type (not a generic type constructor like List on its own).

If A is just Int, tpe is a TypeRepr that represents the scala.Int type.

** tpe.typeSymbol**: From the type representation, we get its primary Symbol. If A is Person, sym is the Symbol for the Person class definition itself. Symbol Represents a declaration or name (e.g., the name of a class User, a method getName, or a field age). It's the "what is it called?" and "where was it defined?".

Line 2: val sym = tpe.typeSymbol

This line is about finding the definition or declaration that created the type.

• tpe: This is the TypeRepr we got from the first line.
• .typeSymbol: This method is called on a TypeRepr. It navigates from the type to the symbol that declared it.
• Symbol: A Symbol in Scala 3 reflection represents a named declaration in your source code. This could be a class, trait, object, method, val, var, or type parameter. It's the "nameplate" of the entity.

What sym contains:

If tpe represents List[String], then sym is the Symbol for the declaration of class List in the Scala library. It has forgotten about the [String] part.

If tpe represents a case class User(name: String), then sym is the Symbol for class User

Step 2: Finding the Fields

val fields: List[Symbol] =
  if (sym.isClassDef) sym.caseFields.toList
  else Nil

• sym.isClassDef: We check if the symbol represents a class. This is a safety measure to ensure we don't try to get fields from a type like Int or String.
• sym.caseFields: This is a powerful helper method on a class Symbol. It returns a List[Symbol] representing the public constructor parameters of the case class. For case class Person(name: String, age: Int), this will be a list containing the symbols for name and age.

Step 3: Generating the Map Entries (The Core Loop)

This is the heart of the macro, where quoting and splicing dance together.

val mapEntries: List[Expr[(String, Any)]] = fields.map { field =>
  val fieldNameExpr = Expr(field.name)
  val fieldAccess = Select(instance.asTerm, field)
  val fieldValueExpr = fieldAccess.asExprOf[Any]
  '{ ($fieldNameExpr, $fieldValueExpr) }
}

We are iterating over each field Symbol to create a quoted tuple ("fieldName", fieldValue).

• val fieldNameExpr = Expr(field.name):

  • field.name gives us the actual string name of the field (e.g., "name").
  • Expr(...) is a helper that quotes a value. It takes a runtime value (the string "name") and creates a quoted expression representing that value's literal code. So Expr("name") produces the AST for the string literal "name".

• val fieldAccess = Select(instance.asTerm, field):

  • This is pure, low-level AST construction.
  • instance.asTerm: We unwrap our type-safe Expr[A] to get the raw Term AST node.
  • Select(..., ...): This is a constructor for a field access AST node. It's equivalent to writing the code instance.field. The first argument is the "qualifier" (the object we're accessing the field on), and the second is the Symbol of the field to access. This creates the AST for person.name.

• val fieldValueExpr = fieldAccess.asExprOf[Any]

  • We take the raw Term AST for person.name and wrap it back up into a type-safe Expr. We cast it to Any because a Map[String, Any] can hold any value type.

• { ($fieldNameExpr, $fieldValueExpr) }: This is the masterpiece.

  • The outer '{ ... } is a Quote. It means: "Do not execute the code inside me now. Instead, construct an AST representing the code inside me."
  • The inner $fieldNameExpr and $fieldValueExpr are Splices. They mean: "Take the quoted expression I have in this variable and splice it into the code I am currently quoting."
  • So, if fieldNameExpr is the AST for "name" and fieldValueExpr is the AST for person.name, this single line builds the AST for the tuple expression ("name", person.name).

The map function returns a List[Expr[(String, Any)]], which is a list of AST nodes, one for each (key, value) pair in our desired map.

Step 4: Assembling the Final Map

'{ Map(${Varargs(mapEntries)}*) }

This is the final step, where we assemble all the tuple ASTs into a single Map AST.

• '{ ... }: Once again, a Quote to say "generate the code for..."
• Map(...): We're generating a call to the Map.apply factory method.

• ${Varargs(mapEntries)}*: This is a special splice.

  • mapEntries is our List[Expr[(String, Any)]].
  • Varargs(...) is a helper that takes a list of quoted expressions and prepares them to be used as a variable argument list. It bundles them into a single Expr[Seq[T]].
  • The splice ${...} injects this generated sequence of expressions into the Map call.
  • The ***** is part of the generated code, not the macro code. It's the syntax for splatting a sequence into varargs

So, this final line generates the complete AST for: Map(("name", person.name), ("age", person.age), ...).

Putting It All Together: A Full Example

Let's trace Person("Alice", 30).toMap.

• Call Site: The compiler sees Person("Alice", 30).toMap.
• Inline Invocation: Because toMap is inline, it calls toMapImpl.

• Parameters:

  • instance is the quoted code '{ Person("Alice", 30) }.
  • A is Person.

• Inside toMapImpl:

  • fields becomes a list of the Symbols for name and age.
  • For name:
    • fieldNameExpr becomes '{ "name" }.
    • fieldAccess becomes the AST for Select(Person("Alice", 30), name_symbol).
    • fieldValueExpr becomes '{ Person("Alice", 30).name }.
    • The map produces the quoted tuple '{ ("name", Person("Alice", 30).name) }.
  • For age:
    • The map produces the quoted tuple '{ ("age", Person("Alice", 30).age) }.

• mapEntries is now a list containing these two quoted tuples.

• Final Code Generation: The last line '{ Map(${Varargs(mapEntries)}*) } produces the final quoted code:

• '{ Map(("name", Person("Alice", 30).name), ("age", Person("Alice", 30).age)) }

• Splicing Back: The compiler takes this result and splices it back into the user's code, replacing the original .toMap call.

• Final Compiled Code: The program now effectively contains:
  
  Map(("name", "Alice"), ("age", 30))

The magic is complete. The boilerplate is gone, the runtime overhead is zero, and the code is perfectly type-safe.

the full source can be found in below gist:
https://gist.github.com/depareddy/06b9f12afc73de3bce71a72bee4d44eb

Permalink

Working with Cloud Optimized GeoTIFFs

Permalink

Initial impressions of Picat

As a tourist into the world of logic programming, I was interested in reading Hillel Wayne talk about Picat, a new (relatively speaking) logical programming language derived from Prolog, but with affordances for functional and imperative programming¹. Of particular note, Picat predicates default to deterministic and you have to explicitly opt into nondetermnism, which is a big change from Prolog, which – as a Prolog beginner – is maybe the thing I find most difficult about writing real programs.

I’ve been looking for something to trial Picat with, and this week I gave it a crack on some Advent of Code puzzles (more about that in a separate post maybe). This experiment resulted in around 400 lines of Picat code (including whitespace and some comments), covering some basic IO processing, algorithms, and building my own little test runner over about 5 days. I have not yet experimented in any real way with the planner or constraint programming libraries.

Picat is clearly a research language. For one thing Github doesn’t have syntax highlighting for it (and there are many hundreds of syntaxes supported there). There is a basic VSCode mode that I suspect is a fork(?) of the Prolog mode: it exhibits the same extremely frustrating bug that occasionaly causes random lines of code from elsewhere in the file to be randomly inserted instead of what you typed when some syntax errors occur.

There are very little in the way of developer ergonomics. Error messages report almost zero context, no line numbers, or stack traces. There’s a basic built in debugger but I leaned on println debugging more. There is a basic REPL, although it doesn’t have the readline style ergonomics you might be used to.

The documentation is comprehensive but terse, and I hope you area okay with Times New Roman for everything (everything even inline syntax). Good news, because the language is fundamentally ungoogleable².

Picat encourages using a functional via it’s API design, and language features. It’s functional programming heavily flavoured by the Prolog origins, so powerful pattern matching and logical variables everywhere. This certainly got me productive quickly: I was writing productive code almost immediately. But it comes at a cost: Picat is not a great functional language, the syntax is, somewhat necessarily, clumsier than an ML or a lisp, or even a modern braces language like Javascript. And the inclusion of functions means that structures are just a little clumsier to work with in Picat than in Prolog: it’s not much but that friction adds up.

Similarly, the imperative facilities are useful (it certainly simplified the IO code I needed to write for Advent of Code), but an expressive imperative language this is not. It reminds me of basic in the 90s. Speaking of which, the IO library exists but is very bare bones.

A consequence of these feature’s is that I found myself writing more functional code than logical code, which was a disappointment. I didn’t really feel like I was getting the advantage of using the language.

As mentioned at the top, I ended up writing a minimal test case runner that would allow you to provide a predicate goal (predicates only, no functions), the name of the variable under test, and the expected value, and it would list successes, failures (predicates with no solutions), and incorrect results. Why did I do this? Because I couldn’t find a testing library in the standard library or elsewhere online.

This was a fun challenge. Prolog, and Picat as a result, have an interesting approach to metaprogramming that feels a bit like lisp macros on rocket fuel. I ended up learning how to make a function that would walk a structure replacing occurances of the named variable under test with a fresh variable, and the executing that goal with the call predicate.

Of course the very first Advent of Code puzzle I tried this on ended up using bigints which immediately blew up the test running (I suspect somewhere deep in the structure rewriting code). Oh well.

I’m not ready to say that I’m finished experienting with this language. But in the 5 days I spent with it, I certainly experienced enough friction to need a break, and ultimately I think Prolog is still probably more interesting to me at this point.

My next logic programming excursion is probably returning to Prolog. SWI Prolog recently released version 10, with improved support for their WASM runtime to let you run it in the browser. I believe Scryer Prolog also has a WASM build. Perhaps I’ll make a Svelte app that wraps up SWI Prolog; I’ve wanted to deploy logical programs for some RPG programs in the browser in the past (Manticore started life as a miniKanren program in Clojure that was rewritten into Typescript).

1. These are basically syntactic sugar: Picat compiles these down to canonical logical forms.
2. Googleʻs search AI will insist on generating answers to your queries that are pure fabrication: some sort of bullshit hybrid of Python and Prolog. It’s almost funny given the confidence it supplies this nonsense with.

Permalink

Understanding Probabilistic Computing with Clojure

Code

;; probablistic_computing.clj

(defn rand-binary [num temp]
  (take num (repeatedly #(< (rand) temp))))

(defn true-count [seq]
  (get (frequencies seq) true))

(true-count (rand-binary 1000 0.7))

(defn x [seq1 seq2]
  (map #(and %1 %2) seq1 seq2))

;; small example

(def small-count 10)

(def small-seventy-percent (rand-binary small-count 0.7))

small-seventy-percent

(true-count small-seventy-percent)

(def small-thirty-percent (rand-binary small-count 0.3))

small-thirty-percent

(true-count small-thirty-percent)

(x small-seventy-percent small-thirty-percent)

(true-count
 (x small-seventy-percent small-thirty-percent))


;; large example

(def large-count 10000)

(def seventy-percent (rand-binary large-count 0.7))

(def thirty-percent (rand-binary large-count 0.3))

(true-count seventy-percent)
(true-count thirty-percent)

(true-count
 (x seventy-percent thirty-percent))

Permalink