How to handle a large number of strings using java?

Question

The processing of each string is: split => encrypt some fields => remove some fields => merge.

First of all, this process of splitting and merging is extremely time-consuming. Can it be processed in batches or not split or other algorithms exist?
Secondly, when the data is transmitted, it is byte[], and it needs to be converted to String during processing, and it must be converted back to byte[] after being merged(Of course, the encryption process also needs to be converted to byte[]).Is there a way to split byte[] with specific characters like String#split()?

Thanks.

Use parallel stream? If you'd show some code, could give an example — Ward
– Ward, Commented Dec 17, 2019 at 13:34
depending on string structure, and how much you actually want to optimize, I'd give a finite automaton a thought; singlepass + data locality should give decent benefit. But it's much more hurdle to implement than using standard lib like String.split, so you really would have to think if you want to bother with such approach — Coderino Javarino
– Coderino Javarino, Commented Dec 17, 2019 at 13:44
I think the optimization point should be on a single piece of data, rather than using parallel streams @Ward — Alsace
– Alsace, Commented Dec 17, 2019 at 14:19

Ward · Accepted Answer · 2019-12-17 13:37:58Z

1

Assuming you input is a Collection<String>, you can probably benefit from a parallelStream

Collection<String> output = input.parallelStream()
    .map(processString())
    .collect(toList());

answered Dec 17, 2019 at 13:37

Ward

2,82823 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1