1

Given a directory, my application traverses and loads .mdb MS Access dbs using the Jackcess API. Inside of each database, there is a table named GCMT_CMT_PROPERTIES with a column named cmt_data containing some text. I also have a Mapper object (which essentially resembles a Map<String,String> but allows duplicate keys) which I use as a dictionary when replacing a certain word from a string.

So for example if mapper contains fox -> dog then the sentence: "The fox jumps" becomes "The dog jumps".

The design I'm going with for this program is as follows:

1. Given a directory, traverse all subdirectories and load all .mdb files into a File[].
2. For each db file in File[], create a Task<Void> called "TaskMdbUpdater" and pass it the db file. 
3. Dispatch and run each task as it is created (see 2. above).

TaskMdbUpdater is responsible for locating the appropriate table and column in the db file it was given and iteratively running a "find & replace" routine on each row of the table to detect words from the dictionary and replace them (as shown in example above) and finally updating that row before closing the db. Each instance of TaskMdbUpdater is a background thread with a Jackcess API DatabaseBuilder assigned to it, so it is able to manipulate the db.

In the current state, the code is running without throwing any exceptions whatsoever, however when I "manually" open the db through Access and inspect a given row, it appears to not have changed. I've tried to pin the source of the issue without any luck and would appreciate any support. If you need to see more code, let me know and I'll update my question accordingly.

public class TaskDatabaseTaskDispatcher extends Task<Void> {

    private String parentDir;
    private String dbFileFormat;
    private Mapper mapper;

    public TaskDatabaseTaskDispatcher(String parent, String dbFileFormat, Mapper mapper) {
        this.parentDir = parent;
        this.dbFileFormat = dbFileFormat;
        this.mapper = mapper;
    }


    @Override
    protected Void call() throws Exception {

        File[] childDirs = getOnlyDirectories(getDirectoryChildFiles(new File(this.parentDir)));
        DatabaseBuilder[] dbs = loadDatabasesInParent(childDirs);

        Controller.dprint("TaskDatabaseTaskDispatcher", dbs.length + " databases were found in parent directory");
        TaskMdbUpdater[] tasks = new TaskMdbUpdater[dbs.length];
        Thread[] workers = new Thread[dbs.length];
        for(int i=0; i<dbs.length; i++) {
            // for each db, dispatch Task so a worker can update that db.
            tasks[i] = new TaskMdbUpdater(dbs[i], mapper);
            workers[i] = new Thread(tasks[i]);
            workers[i].setDaemon(true);
            workers[i].start();
        }

        return null;
    }

    private DatabaseBuilder[] loadDatabasesInParent(File[] childDirs) throws IOException {
        DatabaseBuilder[] dbs = new DatabaseBuilder[childDirs.length];

        // Traverse children and load dbs[]
        for(int i=0; i<childDirs.length; i++) {

            File dbFile = FileUtils.getFileInDirectory(
                    childDirs[i].getCanonicalFile(),
                    childDirs[i].getName() + this.dbFileFormat);

            dbs[i] = new DatabaseBuilder(dbFile);
        }
        return dbs;
    }


} 


// StringUtils class, utility methods
public class StringUtils {

public static String findAndReplace(String str, Mapper mapper) {
        String updatedStr = str;
        for(int i=0; i<mapper.getMappings().size(); i++) {
            updatedStr = updatedStr.replaceAll(mapper.getMappings().get(i).getKey(), mapper.getMappings().get(i).getValue());
        }

        return updatedStr;
    }
}

// FileUtils class, utility methods:
public class FileUtils {
/**
     * Returns only directories in given File[].
     * @param list
     * @return
     */
    public static File[] getOnlyDirectories(File[] list) throws IOException, NullPointerException {
        List<File> filteredList = new ArrayList<>(); 
        for(int i=0; i<list.length; i++) {
            if(list[i].isDirectory()) {
                filteredList.add(list[i]);
            }
        }

        File[] correctSizeFilteredList = new File[filteredList.size()];
        for(int i=0; i<filteredList.size(); i++) {
            correctSizeFilteredList[i] = filteredList.get(i);
        }

        return correctSizeFilteredList;
    }

/**
     * Returns a File[] containing all children under specified parent file.
     * @param parent
     * @return
     */
    public static File[] getDirectoryChildFiles(File parent) {
        return parent.listFiles();
    }
}

public class Mapper {

    private List<aMap> mappings;

    public Mapper(List<aMap> mappings) {
        this.mappings = mappings;
    }

    /**
     * Returns mapping dictionary, typically used for extracting individual mappings.
     * @return List of type aMap 
     */
    public List<aMap> getMappings() {
        return mappings;
    }

    public void setMappings(List<aMap> mappings) {
        this.mappings = mappings;
    }
} 

/**
 * Represents a single String based K -> V mapping.
 */
public class aMap {

    private String[] mapping; // [0] - key, [1] - value

    public aMap(String[] mapping) {
        this.mapping = mapping;
    }

    public String getKey() {
        return mapping[0];
    }

    public String getValue() {
        return mapping[1];
    }

    public String[] getMapping() {
        return mapping;
    }

    public void setMapping(String[] mapping) {
        this.mapping = mapping;
    }
} 

Update 1: To verify my custom StringUtils.findAndReplace logic, I've performed the following unit test (in JUnit) which is passing:

@Test
    public void simpleReplacementTest() {
        // Construct a test mapper/dictionary
        List<aMap> aMaps = new ArrayList<aMap>();
        aMaps.add(new aMap(new String[] {"fox", "dog"})); // {K, V} = K -> V
        Mapper mapper = new Mapper(aMaps);

        // Perform replacement
        String corpus = "The fox jumps";
        String updatedCorpus = StringUtils.findAndReplace(corpus, mapper);
        assertEquals("The dog jumps", updatedCorpus);
    }

I'm including my TaskMdbUpdater class here separately with some logging code included, as I suspect point of failure lies somewhere in call:

/**
 * Updates a given .mdb database according to specifications defined internally.
 * @since 2.2
 */
public class TaskMdbUpdater extends Task<Void> {

    private final String TABLE_NAME = "GCMT_CMT_PROPERTIES";
    private final String COLUMN_NAME = "cmt_data";

    private DatabaseBuilder dbPackage;
    private Mapper mapper;

    public TaskMdbUpdater(DatabaseBuilder dbPack, Mapper mapper) {
        super();
        this.dbPackage = dbPack;
        this.mapper = mapper;
    }

    @Override
    protected Void call() {
        try {
//      Controller.dprint("TaskMdbUpdater", "Worker: " + Thread.currentThread().getName() + " running");

        // Open db and extract Table
        Database db = this.dbPackage
                .open();
        Logger.debug("Opened database: {}", db.getFile().getName());

        Table table = db.getTable(TABLE_NAME);
        Logger.debug("Opening table: {}", table.getName());

        Iterator<Row> tableRows = table.iterator();

//      Controller.dprint("TaskMdbUpdater", "Updating database: " + db.getFile().getName());

        int i=0;
        try {
            while( tableRows.hasNext() ) {

                // Row is basically a<code> Map<Column_Name, Value> </code>
                Row cRow = tableRows.next();
                Logger.trace("Current row: {}", cRow);

//              Controller.dprint(Thread.currentThread().getName(), "Database name: " + db.getFile().getName());
//              Controller.dprint("TaskMdbUpdater", "existing row: " + cRow.toString());
                String str = cRow.getString(COLUMN_NAME);
                Logger.trace("Row {} column field contents (before find/replace): {}", i, str);

                String newStr = performFindAndReplaceOnString(str);
                Logger.trace("Row {} column field contents (after find/replace): {}", i, newStr);

                cRow.put(COLUMN_NAME, newStr);
                Logger.debug("Updating field in row {}", i);

                Row newRow = table.updateRow(cRow); // <code>updateRow</code> returns the new, updated row. Ignoring this.
                Logger.debug("Calling updateRow on table with modified row");

//              Controller.dprint("TaskMdbUpdater", "new row: " + newRow.toString());
                i++;
                Logger.trace("i = {}", i);
            }
        } catch(NoSuchElementException e) {
//          e.printStackTrace();
            Logger.error("Thread has iterated past number of rows in table", e);
        }
        Logger.info("Iterated through {} rows in table {}", i, table.getName());

        db.close();
        Logger.debug("Closing database: {}", db.getFile().getName());

        } catch (Exception e) {
//          e.printStackTrace();
            Logger.error("An error occurred while attempting to update row value", e);
        }
        return null;
    }

    /**
     * @see javafx.concurrent.Task#failed()
     */
    @Override
    protected void failed() {
        super.failed();
        Logger.error("Task failed");
    }

    @Override
    protected void succeeded() {
        Logger.debug("Task succeeded");
    }

    private String performFindAndReplaceOnString(String str) {
//      Logger.trace("OLD: [" + str + "]");

        String updatedStr = null;
        for(int i=0; i<mapper.getMappings().size(); i++) {
            // loop through all parameter names in mapper to search for in str.
            updatedStr = findAndReplace(str, this.mapper);
        }
//      Logger.trace("NEW: [" + updatedStr + "]");
        return updatedStr;
    }
}

Here's a small exerept from my log. As you can see, it doesn't seem to do anything after opening the table which has left me a bit perplexed:

INFO (16-02-2017 17:27:59) [Thread-9] NAMEMAP.logic.TaskDatabaseTaskDispatcher.call(): Located the following directories under specified MOIS parent which contains an .mdb file:
[01_Parent_All_Safe_Test[ RV_DMS_0041RV_DMS_0001RV_DMS_0003RV_DMS_0005RV_DMS_0007RV_DMS_0012RV_DMS_0013RV_DMS_0014RV_DMS_0016RV_DMS_0017RV_DMS_0018RV_DMS_0020RV_DMS_0023RV_DMS_0025RV_DMS_0028RV_DMS_0029RV_DMS_0031RV_DMS_0033RV_DMS_0034RV_DMS_0035RV_DMS_0036RV_DMS_0038RV_DMS_0039RV_DMS_0040 ]]
...
DEBUG (16-02-2017 17:27:59) [Thread-9] NAMEMAP.logic.TaskDatabaseTaskDispatcher.call(): Created new task: NAMEMAP.logic.TaskMdbUpdater@4cfe46fe
DEBUG (16-02-2017 17:27:59) [Thread-9] NAMEMAP.logic.TaskDatabaseTaskDispatcher.call(): Created new worker: Thread[Thread-22,5,main]
DEBUG (16-02-2017 17:27:59) [Thread-9] NAMEMAP.logic.TaskDatabaseTaskDispatcher.call(): Set worker Thread[Thread-22,5,main] as daemon
DEBUG (16-02-2017 17:27:59) [Thread-9] NAMEMAP.logic.TaskDatabaseTaskDispatcher.call(): Dispatching worker: Thread[Thread-22,5,main]
...
DEBUG (16-02-2017 17:28:00) [Thread-22] NAMEMAP.logic.TaskMdbUpdater.call(): Opened database: RV_DMS_0023.mdb
DEBUG (16-02-2017 17:28:00) [Thread-22] NAMEMAP.logic.TaskMdbUpdater.call(): Opening table: GCMT_CMT_PROPERTIES

After this point, there isn't any more entries entries in the log and the processor spikes at 100% load, remaining that way until I force kill the application. This could mean the program gets stuck in an infinite while loop - however if that were to be the case then shouldn't there be log entries in the file?

Update 2

Okay I've further narrowed the problem by printing log TRACE into stdio. It seems that my performFindAndReplaceOnString is super inefficient and it never gets past the first row of these dbs because it's just grinding away at the long string. Any suggestions on how I can efficiently perform a string replacement for this use case?

7
  • Are you making sure that you .close() the Database object properly before your application exits? Commented Feb 15, 2017 at 20:12
  • I'd like to believe so. If I can draw your attention to call method of class TaskMdbUpdater, you can see I'm closing the Database object that I opened at the beginning of that method. ... catch(NoSuchElementException e) { e.printStackTrace(); } db.close(); } catch (Exception e) { e.printStackTrace(); } return null; Commented Feb 16, 2017 at 9:37
  • have you confirmed that your findAndReplace logic is actually correctly modifying the strings? Commented Feb 16, 2017 at 16:39
  • Yes, my unit test is passing, see update 1 above ^ Commented Feb 16, 2017 at 18:12
  • 1
    "Any suggestions on how I can efficiently perform a string replacement for this use case?" - Some ideas here. Commented Feb 16, 2017 at 19:02

0

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.