Focused crawls are collections of frequently-updated webcrawl data from narrow (as opposed to broad or wide) web crawls, often focused on a single domain or subdomain.
For an implementation of #126 (PostgreSQL driver with SKIP LOCKED), I create a SQL table for each consumer group containing the offsets ready to be consumed. The name for these tables is build by concatenating some prefix, the name of the topic and the name of the consumer group. In some of the test cases in the test suite, UUID are used for both, the topic and the consumer group. Each UUID has
The SubscriberDeliveryTaskCount default should be set to something sane like 3. Currently it is zero which is too strongly biased toward high throughput with a low number of subscriptions and does not support many subscriptions in one application as one thread to be created per subscr
We should create an integration test that splits up a big file or it performs an expected mathematical operations on generating records, in a way that the output can be inferred and checked.
e.g.:
splits up a 100MB video file into several 1MB files, sends to the commit log, re-assemble the file and checks md5 of the original and the re-assembled.
For an implementation of #126 (PostgreSQL driver with
SKIP LOCKED), I create a SQL table for each consumer group containing the offsets ready to be consumed. The name for these tables is build by concatenating some prefix, the name of the topic and the name of the consumer group. In some of the test cases in the test suite, UUID are used for both, the topic and the consumer group. Each UUID has