Change location of `~/.python_history`

Question

I want to keep the functionality, but change the location of ~/.python_history to $XDG_DATA_HOME/python/python_history.

This gave me the following idea: I could create $XDG_CONFIG_HOME/python/pythonrc and point $PYTHONSTARTUP to it. In there I want to replace the functions readline.read_history_file, readline.write_history_file and readline.append_history_file.

Is there way to replace these functions with themselves, containing a custom filename argument?

If not, do you have another idea how to approach this?

Answer 1

This is possible in Python 3.13 with the merge of PR 13208. It added an environment variable PYTHON_HISTORY that can be used to customize the history file location.

export PYTHON_HISTORY=~/.local/share/python/history

Answer 2

Like you & ctrl-alt-delor, I sought the mythical clean home directory. The call to readline.write_history_file is registered to run on exit in site.py (on my arch system, at /usr/lib/python3.9/site.py):

if readline.get_current_history_length() == 0:
    # If no history was loaded, default to .python_history.
    # The guard is necessary to avoid doubling history size at
    # each interpreter exit when readline was already configured
    # through a PYTHONSTARTUP hook, see:
    # http://bugs.python.org/issue5845#msg198636
    history = os.path.join(os.path.expanduser('~'),
                           '.python_history')
    try:
        readline.read_history_file(history)
    except OSError:
        pass

    def write_history():
        try:
            readline.write_history_file(history)
        except OSError:
            # bpo-19891, bpo-41193: Home directory does not exist
            # or is not writable, or the filesystem is read-only.
            pass

    atexit.register(write_history)

You can duplicate this in your PYTHONSTARUP but with a custom location for the python history. Here's mine (but if you like I'm sure you could used environment substitution to use the appropriate XDG directory.:

import os
import atexit
import readline

history = os.path.join(os.path.expanduser('~'), '.cache/python_history')
try:
    readline.read_history_file(history)
except OSError:
    pass

def write_history():
    try:
        readline.write_history_file(history)
    except OSError:
        pass

atexit.register(write_history)

You could overwrite the write_history_file function, but it's very hacky (you need it to ignore the argument it's given in lieu of your custom one), so I think this is the best solution. If this doesn't work, try creating a dummy entry in your custom python history file, so the history length is greater than 0.

Answer 3

The one xdg-ninja uses (pasting here for convenience):

Export environment variable:

export PYTHONSTARTUP="/etc/python/pythonrc"

Create /etc/python/pythonrc:

import os
import atexit
import readline
from pathlib import Path

if readline.get_current_history_length() == 0:
    state_home = os.environ.get("XDG_STATE_HOME")
    if state_home is None:
        state_home = Path.home() / ".local" / "state"
    else:
        state_home = Path(state_home)

    history_path = state_home / "python_history"
    if history_path.is_dir():
        raise OSError(f"'{history_path}' cannot be a directory")

    history = str(history_path)

    try:
        readline.read_history_file(history)
    except OSError:  # Non existent
        pass

    def write_history():
        try:
            readline.write_history_file(history)
        except OSError:
            pass

    atexit.register(write_history)

Answer 4

None of the other answers here prevent ~/.python_history from being created / written to.

cpython registers an __interactivehook__ for the function register_readline here: https://github.com/python/cpython/blob/main/Lib/site.py#L485

This function sets up interpreter tab-completion, as well as a hook to write the history to ~/.python_history when the interpreter exits. We can delete this hook to prevent ~/.python_history from being written. We can then set up tab completion manually and register our own history writing hook.

Place the following code in your $PYTHONSTARTUP file:

try:
    import atexit
    import os
    import sys
    from pathlib import Path
    import readline
except ImportError as e:
    print(f"Couldn't load module. {e}")
    sys.exit(1)


################
# TAB COMPLETION #
##################

try:
    readline.parse_and_bind("tab: complete")
except ImportError:
    pass


### XDG Compliant History File
# See https://gist.github.com/viliampucik/8713b09ff7e4d984b29bfcd7804dc1f4?permalink_comment_id=4582040#gistcomment-4582040

# Destroy default history file writing hook (and also tab completion, which is why we manually added it above)
if hasattr(sys, '__interactivehook__'):
    del sys.__interactivehook__


histfile = Path(os.getenv("XDG_CACHE_HOME", Path.home() / ".cache")) / "python_history"
try:
    histfile.touch(exist_ok=True)
except FileNotFoundError: # Probably the parent directory doesn't exist
    histfile.parent.mkdir(parents=True, exist_ok=True)

readline.read_history_file(histfile)
# Don't store an obscene amount of history
readline.set_history_length(5000)
# Write to history file on exit
atexit.register(readline.write_history_file, histfile)

Run $ mv ~/.python_history $XDG_CACHE_HOME/python_history and then run $ python3 and enter a command. You should expect that ~/.python_history was not created, and $XDG_CACHE_HOME/python_history contains your updated history.

Answer 5

My take on this, inspired by other answers:

# Enable custom ~/.python_history location on Python interactive console
# Set PYTHONSTARTUP to this file on ~/.profile or similar for this to work
# https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSTARTUP
# https://docs.python.org/3/library/readline.html#example
# https://github.com/python/cpython/blob/main/Lib/site.py @ enablerlcompleter()
# https://unix.stackexchange.com/a/675631/4919

import atexit
import os
import readline
import time


def write_history(path):
    import os
    import readline
    try:
        os.makedirs(os.path.dirname(path), mode=0o700, exist_ok=True)
        readline.write_history_file(path)
    except OSError:
        pass


history = os.path.join(os.environ.get('XDG_CACHE_HOME') or
                       os.path.expanduser('~/.cache'),
                       'python_history')
try:
    readline.read_history_file(history)
except FileNotFoundError:
    pass

# Prevents creation of default history if custom is empty
if readline.get_current_history_length() == 0:
    readline.add_history(f'# History created at {time.asctime()}')

atexit.register(write_history, history)
del (atexit, os, readline, time, history, write_history)

Answer 6

I had the same annoyance of lots of files in my home directory. This is what I did.

• Create a directory ~/+Files.
• Move all of my(not the annoying config files) files to this new directory.
• Configure interactive shell, and dolphin to start in this new directory.
• Create a directory for config files ~/+Files/config
• Move config files to this new directory, and sym-link to them.

Now my home directory contains only sym-links and +Files directory. From time-to-time I then inspect my home directory for cruft that has turned up. And decide what to do with it: move to config, remove, …