Create Task
Maniphest T425897

Future of filearchive table
Open, Needs TriagePublic
Actions

Description

T28741: Migrate file tables to a modern layout (image/oldimage; file/filerevision; add primary keys) does not yet touched the filearchive table. One may expect this table should also be migrated to filerevision, but it depends on how we will do in T20493: RFC: Unify the various deletion systems. This task is just a placeholder before the two subtasks are resolved.

Even if we decided to keep this table, some optimization can be done, such as:

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenNoneT425897 Future of filearchive table
 OpenLadsgroupT28741 Migrate file tables to a modern layout (image/oldimage; file/filerevision; add primary keys)
 ResolvedKrinkleT589 RFC: image and oldimage tables
 ResolvedLadsgroupT368113 Design and merge the new tables of file tables
 ResolvedmfornsT389800 Update Commons Impact Metrics to account for new File table
 ResolvedLadsgroupT383093 Set beta cluster to write both in file schema and thoroughly test it
 ResolvedLadsgroupT383491 Deploy new file tables in production
 ResolvedLadsgroupT383492 Write the migration script
 ResolvedZabeT383496 Add support for reading new file schema into MediaWiki
 ResolvedDreamy_JazzT383555 Add support for reading new file schema in MediaModeration
 ResolvedLadsgroupT384481 Set new file tables to write both in production
 ResolvedLadsgroupT385167 Run data migration script for file migration
 DuplicateNoneT386133 Update Data Pipelines with new image table schema
 ResolvedZabeT412164 Set beta cluster to read new in file schema and thoroughly test it
 ResolvedLadsgroupT415330 Add support for stop writing to the old file db schema
 OpenZabeT416548 Start reading from file table on wmf production
 OpenPRODUCTION ERRORNoneT418007 Wikimedia\Rdbms\DBQueryError: Error 1062: Duplicate entry 'Savannah_River_(26769689014).jpg' for key 'file_name'Function: MediaWiki\FileRepo\File\LocalFile::acquireFileIdFromNameQuery: INSERT INTO `file` (file_name,file_lates
 ResolvedPRODUCTION ERRORZabeT418327 PHP Deprecated: Caller from MediaWiki\Exception\MWExceptionHandler::rollbackPrimaryChanges ignored an error originally raised from MediaWiki\Pager\IndexPager::buildQueryInfo (MediaWiki\Specials\Pager\ImageListPager): [1054] Unk
 ResolvedPRODUCTION ERRORZabeT419062 Wikimedia\Rdbms\DBQueryError: Error 1054: Unknown column 'img_timestamp' in 'WHERE'Function: MediaWiki\Pager\IndexPager::buildQueryInfo (MediaWiki\Specials\Pager\NewFilesPager)Query: SELECT /*! STRAIGHT_JOIN */ file_name AS
 ResolvedPRODUCTION ERRORZabeT422946 Expectation (readQueryTime <= 5) by MediaWiki\Actions\ActionEntryPoint::execute not met (actual: {actualSeconds}) in trx #{trxId}:{query}
 ResolvedSecurityZabeT423654 Expectation (readQueryTime <= 5) by MediaWiki\Actions\ActionEntryPoint::execute not met (actual: {actualSeconds}) in trx #{trxId}:{query}
 ResolvedBUG REPORTLadsgroupT423978 Special:Mediastatistics is being weird and not grouping similar file types together
 OpenZabeT425191 Drop old image tables from wikireplicas
 ResolvedZabeT426804 filerevision view should not filter out deleted file revisions
 StalledZabeT428970 Stop writing to the old file db schema
 ResolvedBUG REPORTManvikesarwani09T429720 Files on Betacommons can be moved once, then never again
 OpenNoneT20493 RFC: Unify the various deletion systems

Event Timeline

Bugreporter created this task.May 11 2026, 1:59 AM
Restricted Application added a project: Data-Engineering. · View Herald TranscriptMay 11 2026, 1:59 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Bugreporter added subtasks: T28741: Migrate file tables to a modern layout (image/oldimage; file/filerevision; add primary keys), T20493: RFC: Unify the various deletion systems.May 11 2026, 2:00 AM
Bugreporter mentioned this in T425893: MediaWiki is incapable of deleting files with too many versions.
Maintenance_bot added a project: Commons.May 11 2026, 2:30 AM
Bugreporter updated the task description. (Show Details)May 12 2026, 2:39 AM
Ottomata moved this task from Incoming (new tickets) to Backlog on the Data-Engineering board.Jun 3 2026, 6:56 PM
Ottomata moved this task from Backlog to Incoming (new tickets) on the Data-Engineering board.
Krinkle mentioned this in T428296: Change "shorturl" in FileRepo to something searchengine-friendly.Jun 7 2026, 2:18 AM
jcrespo added subscribers: Ladsgroup, jcrespo.Jun 17 2026, 12:49 PM

@Ladsgroup Should I keep scanning filearchive table to backup deleted files for now? My understanding is currently yes.

Ladsgroup added a comment.Jun 17 2026, 1:07 PM

Yes. filearchive is not going to be changed at all in the foreseeable future. I don't think the subtasks here are correctly honestly.

Bugreporter added a comment.Jun 17 2026, 4:20 PM

One of reasons of T20493: RFC: Unify the various deletion systems is deleting and undeleting pages with many revisions involves moving rows between tables, which is much slower than just update some fields (e.g. moving rows will have larger impact on indices). Previously deleting pages with many revisions is troublesome; T198176: Mediawiki page deletions should happen in batches of revisions mitigates it, but we does not yet have a similar mitigation for undeletion (T239095: Undeleting pages with many revision results in DB exceptions), nor deleting files with many revisions (T425893: MediaWiki is incapable of deleting files with too many versions).

Note deleting files is a heavier operation than deleting pages, since files need to be moved to new location, and in Swift (used by Wikimedia) files can only be moved via copying and deletion.

Bugreporter updated the task description. (Show Details)Jun 17 2026, 4:27 PM
Ottomata moved this task from Incoming (new tickets) to Tag with Radar on the Data-Engineering board.Jun 17 2026, 5:07 PM
Ottomata added a project: Data-Engineering-Radar.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits