The Wayback Machine - https://web.archive.org/web/20200821231428/https://github.com/dimitri/pgloader/issues/1125
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a statistics singleton #1125

Open
phoe opened this issue Apr 3, 2020 · 4 comments
Open

Introduce a statistics singleton #1125

phoe opened this issue Apr 3, 2020 · 4 comments
Assignees

Comments

@phoe
Copy link
Collaborator

@phoe phoe commented Apr 3, 2020

Originally posted by @dimitri in https://github.com/dimitri/pgloader/timeline

That's good thinking, and I think that's relevant for another PR. The PR introducing the statistics singleton should also take care of the following bits of refactoring of the code:

  • the singleton is named “state” as I initially though I would update the global state of the migration to be able to know about the progress being made while it's being made,
  • when implementing the feature, it quickly enough turned into statistics (count of rows read, imported, errors, timings, etc) rather “state”
  • the API to access and maintain the state objects (stats really) per table etc is rather lacking, so we should clean that up and offer easy access to needed information from threads

At the moment only the monitor thread is creating and updating the data, we certainly want to keep it that way. That allows no-locks (and no dead locks) code, with using a queue to send events and centralise statistics from.

@dimitri
Copy link
Owner

@dimitri dimitri commented Apr 3, 2020

Hey @phoe ; thanks for opening the issue to keep track of it. Do you want to take care of this? it's a pretty well isolated piece of pgloader and would allow you to stay in the Common Lisp side of things without having to know much about the database side of things ;-)

@phoe phoe self-assigned this Apr 3, 2020
@phoe
Copy link
Collaborator Author

@phoe phoe commented Apr 3, 2020

Zrzut ekranu z 2020-04-03 23-29-42

Done, I'll poke it in a spare while.

@simion-iulian
Copy link

@simion-iulian simion-iulian commented May 11, 2020

@phoe @dimitri, I had a look through open issues and PRs and I think I'd like to have a take at this one.
Did any of you already start work on it?

I would not like to duplicate work. If you did and would like to do a handover that would be very welcome.

Thanks!

@phoe
Copy link
Collaborator Author

@phoe phoe commented May 11, 2020

I haven't started any work on it. Please feel free to proceed! Fame and glory awaits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
You can’t perform that action at this time.