Functional architecture with lots of I/O

It is really-really difficult to explain this in a short answer, but you do that by essentially deferring any I/O (any non-pure action) to the "shell", what in Haskell would just be main.

So instead of, for example, querying the database and returning a count of users, you return a database operation that has an integer "value". You can continue to work with this unknown value, but the operation is not really executed. It is only executed once it gets "out" into the outer shell.

Your "interpreter" comment was not that far off, except you don't switch-case in the outer shell, you directly receive an "object" (call it whatever you like) that can be "executed" to do the dirty stuff. Sidenote: there are literally interpreter-based solutions that also exist.

I suggest you start with learning what monads are, if you have not done so yet, and go from there.

Calling it a "shell" and a "core" makes it sound more cohesive than it actually is. The pattern is more that your user management has an IO layer and a pure layer, and your session management has an IO layer and a pure layer, and your API calls have an IO layer and a pure layer, then you kind of have some glue that sticks all the IO layers together. Sometimes people treat pervasive things like logging or random numbers as okay to be impure because they aren't "in the main path," but there are concepts like the writer and state monads to handle those sorts of concerns as well.

This pattern is more and more common in imperative programming too, due to wanting to mock out the IO for testing. If you've ever made a minimal interface with one production implementation that does real IO and a test implementation that returns dummy data, and you push as much logic as possible to code that depends on this interface, then you've done the "impure core" architectural pattern, at least in a small part. Functional programmers just take it a step or two further.

My understanding of "Functional Core, Imperative Shell" is basically that you do all the Input (I) in I/O first in the imperative shell, then you call your pure functions (in the functional core) to generate some result based on the input, and then in the imperative shell you take the result and do the Output (O) in I/O.

This has also been called Impureim sandwich by Mark Seemann.

To simplify things, you have three functions:

• An impure function to read inputs: () => Input
• A pure function: Input => Result
• An impure function to write outputs: Result => ()

What you have pointed out is very true: things are not always linear in this way.

1. You can have a long running process (function) that needs to be fed new inputs while it is running. This can be because of performance reasons, e.g. you cannot read all orders from the database at once. Or it can be for logical reasons, e.g. you don't know which orders you need to get from the database before doing some calculations.
2. You can have a long running process (function) that needs to output some data while it is running. Again, this can be for performance reasons, e.g. you cannot return a 4GB object at the end of the function call, so you need to insert data into the database in pieces while you are running. Or it can be for monitoring reasons, e.g. you want to display status to the user while you are running.
3. Even a short running function might need to get data from different sources based on some logic, e.g. you do some calculations then then decide you want to grab data from source 1 or source 2 based on the results of the calculations, and then you need to do calculations based on the data returned from source 1 or source 2. Based on the result of those calculations you need to read data from source 3 or source 4, and this can go on and on...

I think in those cases (which are very common), you really cannot have a true "Functional Core, Imperative Shell".

I think the next best thing is an Honest Potentially-Functional Core, Imperative Shell.

That is, you create a potentially-functional core. That is a collection of potentially-pure functions.

Basically, your functions are for example:

• An impure function to read inputs: () => Input
• A potentially-pure function: (Input, () => MoreInputs1, () => MoreInputs2, SomeResults1 => (), SomeResults2 => (), SomeResults3 => SomeInputs3) => Result
• An impure function to write outputs: Result => ()

When the imperative shell calls the potentially-pure function, it passes impure functions for the following arguments (which are of type function):

• () => MoreInputs1
• () => MoreInputs2
• SomeResults1 => ()
• SomeResults2 => ()
• SomeResults3 => SomeInputs3

The "pure" thing about your potentially-pure function is that it declares all dependencies that would make it impure. This makes it an honest function.

See this article for more details about composing potentially-pure functions: https://divex.dev/knowledge-base/main/dependency-injection-can-be-functional/

I think there are other ways to create a potentially-functional core. I think the Haskell IO monad is one such way, but I am not sure.

It's kind of hard to answer this sort of question without a concrete example IMO but let's see what I can do:

In order to get a user from the database, you need to create some sort of database command from a user ID (pure code), call the DB using the command (impure code), then parse the response into a User DTO (pure command).

So your traditional "getUser()" function is impure and within it you have a pure function, an impure function, and then another pure function. Then you are going to be passing that user into another impure function that has a pure part, then an impure part, then another pure part. And so on... So the idea of FC/IS is to reverse that notion.

The simplest idea is to reverse your injection system, instead of injecting impure code into your pure code, you inject pure code into your impure code.

Again, the traditional getUser() function accepts (in some way) an interface that represents the impure part that is in the middle. We do this so we can test the pure parts of the function by mocking out the impure part.

Likely, you currently have some sort of class like this:

class UserDB { 
    UserDB(dbInterface) { 
        self.dbInterface = dbInterface
    }
    func getUser(id) { // [impure] 
        command = createGetUserCommand(id) // [pure]
        response = self.dbInterface.perform(command) // [impure]
        return createUserFrom(response) // [pure]
    }
}

Let's reverse that notion...

class GetUserProcess { 
    GetUserProcess(id) { self.id = id }
    func createCommand() // [pure]
    func handleResponse(dbResponse) // [pure]
}

class DBSystem { 
    func perform(process) { // [impure]
        command = process.createCommand()
        response = // do the db access using the command
        return process.handleResponse(response)
    }
}

By reversing the notion and injecting the pure code into your more generic impure code, you can test the pure parts without having to mock the impure part. It also reduces the number of impure functions in the system.

Taking it a step further, your entire code base consists of alternating pure and impure parts... You have likely portioned it out like this:

(pure->impure->pure)->(pure->impure->pure)->(pure->impure->pure)

Each pure->impure->pure bit is a function...

Just change where the function breaks are: (pure)->(impure)->(pure->pure)->(impure)->(pure->pure)->(impure)->(pure)

The pure and impure parts are isolated from each other.

This way, you don't need to constantly mock out the impure parts in order to test the pure parts.