DEV Community

Cover image for Setting up a Python project with CMake
pikoTutorial
pikoTutorial

Posted on • Originally published at pikotutorial.com

Setting up a Python project with CMake

#python #programming #testing #unittest

Welcome to the next pikoTutorial!

CMake is often associated only with C/C++ and occupies a high place in the ranking of the most hated tools. Today, I want to show an unusual, but interesting use case - setting up and running Python applications with CMake and its underlying generators. This can be especially useful in mixed-language projects where production code is written in C++, tools in Python and all that is integrated together with CMake.

Take a look on how to run with a single command any application regardless of the language it is written in or the virtual environment it uses.

Project structure 

project/
├── app1/
│   ├── CMakeLists.txt
│   ├── main.py
│   ├── requirements.txt
├── app2/
│   ├── CMakeLists.txt
│   ├── main.py
│   ├── requirements.txt
├── app3/
│   ├── CMakeLists.txt
│   ├── main.cpp
└── build/
├── CMakeLists.txt
Enter fullscreen mode Exit fullscreen mode

CMakeLists.txt file 

# specify minimu CMake version
cmake_minimum_required(VERSION 3.28)
# specify project name
project(ExamplePythonSetup)
# find Python
find_package(Python3 REQUIRED COMPONENTS Interpreter)
# define a function for creating Python virtual environment
function(create_venv venv_dir requirements_path)
    # check if the virtual environment already exists
    if(EXISTS ${venv_dir})
        message(STATUS "Virtual environment already exists in ${venv_dir}, skipping creation.")
        return()
    endif()
    # ensure that the given requirements.txt file exists
    if(NOT EXISTS ${requirements_path})
        message(FATAL_ERROR "Requirements file not found: ${requirements_path}")
    endif()
    # create the virtual environment
    execute_process(
        COMMAND ${Python3_EXECUTABLE} -m venv ${venv_dir}
        RESULT_VARIABLE venv_creation_ret_code
    )
    # report error if return code is non-zero
    if(venv_creation_ret_code)
        message(FATAL_ERROR "Failed to create virtual environment at ${venv_dir}!")
    endif()
    # install dependencies from requirements.txt
    execute_process(
        COMMAND ${venv_dir}/bin/pip install -r ${requirements_path}
        RESULT_VARIABLE pip_install_ret_code
    )
    # report error if return code is non-zero
    if(pip_install_ret_code)
        message(FATAL_ERROR "Failed to install dependencies from ${requirements_path}!")
    endif()
    # print success message
    message(STATUS "Virtual environment setup done at ${venv_dir} with dependencies from ${requirements_path}")
endfunction()
# include all subdirectoies into the build
add_subdirectory(app1)
add_subdirectory(app2)
add_subdirectory(app3)
Enter fullscreen mode Exit fullscreen mode

app1/CMakeLists.txt 

# specify app1 virtual environment directory
set(APP1_VENV ${CMAKE_BINARY_DIR}/app1_venv)
# create virtual environment for app1
create_venv(${APP1_VENV} ${CMAKE_SOURCE_DIR}/app1/requirements.txt)
# add custom target to run app1
add_custom_target(run_app1
    COMMAND ${APP1_VENV}/bin/python ${CMAKE_SOURCE_DIR}/app1/main.py
    DEPENDS ${APP1_VENV}
)
Enter fullscreen mode Exit fullscreen mode

app2/CMakeLists.txt 

# specify app2 virtual environment directory
set(APP2_VENV ${CMAKE_BINARY_DIR}/app2_venv)
# create virtual environment for app2
create_venv(${APP2_VENV} ${CMAKE_SOURCE_DIR}/app2/requirements.txt)
# add custom target to run app2
add_custom_target(run_app2
    COMMAND ${APP2_VENV}/bin/python ${CMAKE_SOURCE_DIR}/app2/main.py
    DEPENDS ${APP2_VENV}
)
Enter fullscreen mode Exit fullscreen mode

app3/CMakeLists.txt 

# create an executable out of C++ code
add_executable(main main.cpp)
# add custom target to run app3
add_custom_target(run_app3
    COMMAND ${CMAKE_BINARY_DIR}/app3/main
    DEPENDS main
)
Enter fullscreen mode Exit fullscreen mode

Project configuration

In the build folder run:

cmake ..
Enter fullscreen mode Exit fullscreen mode

At this stage CMake will call (among others) execute_process functions which in this case will create Python virtual environments for both Python applications and download the corresponding dependencies specified in their requirements.txt files.

To build all the targets run:

cmake --build .
Enter fullscreen mode Exit fullscreen mode

From now on, you can run every application using make, regardless of whether it's a Python or C++ application or whether it uses one virtual environment or the other:

make run_app1
make run_app2
make run_app3
Enter fullscreen mode Exit fullscreen mode

Or, if you use VS Code, you can now see all the above 3 targets in the bottom bar, so you can run each of them by pressing a button on the CMake GUI interface.

Using CMake and Ninja

If you think that make just doesn't feel right for this use case, you can of course generate configuration based on another tool - Ninja. To do that, invoke:

cmake .. -G Ninja
cmake --build .
Enter fullscreen mode Exit fullscreen mode

Now you can run all the applications calling:

ninja run_app1
ninja run_app2
ninja run_app3
Enter fullscreen mode Exit fullscreen mode

Top comments (0)