How do I parallelise multiple HTTP calls using RxJS

Question

Background

I'm building an Angular application which lists companies that are registered at a given address.

To give some context...

Let's say I have x3 companies: Company A, Company B and Company C.

Company B is registered at the same address as Company A. Company C is not.

When I navigate to the app and filter on Company A, I expect to only see Company B in the list.

Problem

My issue is not that I couldn't get this working, it is simply too slow! I need to leverage multithreading/concurrency somehow.

In order to work out which companies are registered at the given address, I have to make several HTTP calls.

Before explaining the sequence in which I make the HTTP calls. Let me show you what the API looks like:

GET /api/companies/CompanyA/address
{
  id: 1,
  addressLine1: '123 Some street',
  ...
}

GET /api/companies/CompanyA/links
[
  {id: 2, name: 'Company B'},
  {id: 3, name: 'Company C'},
]

Right, here is the sequence:

1. Get the address for Company A and store the ID
2. Get the links for Company A
3. Loop over each link 3a. Get the address of the link 3b. Check if the address ID matches Company A's address ID. If it does, store the link.

Current Implementation

const companyAId: number = 1;    

const companyAAddress: Object = await this.httpService.getAddress(companyAId).toPromise();

const companyALinks: Object[] = await this.httpService.getLinks(companyAId).toPromise();

const companiesToShow: Object[] = [];
for (let link of links) {
    const linkAddress: Object = await this.httpService.getAddress(link ['id']).toPromise();
    if (linkAddress['id'] === companyAAddress['id']) companiesToShow.push(link );
}

There must be a more elegant / performant solution!

Any help will be greatly appreciated :)

Thanks,

Ben

Answer 1

You are correct in that each of these API calls has to wait for the other to complete first.

You don't have access to multithreading per se, but you can make this concurrent using Promise.all as you've written it. Specifically, you can retrieve the address and the links at the same time and then all of the other addresses at the same time. I've shortened some of the method call names to make it easier to write:

const [companyAAddress, companyALinks] = await Promise.all(
  getAddress(id).toPromise(),
  getLinks(id).toPromise(),
);

const companiesToShow = await Promise.all(links.map(link => getAddress(link.id).pipe(
  map(linkAddress => [link, linkAddress]),
  filter(([, { id }]) => id === companyAAddress.id),
).toPromise())

Rather than use promises, you can do this strictly using Observables which will make it cancelable.

forkJoin(getAddress(id), getLinks(id)).pipe(
  mergeMap(([companyAAddress, companyALinks]) => companyALinks.map(link => getAddress(link.id).pipe(
    map(linkAddress => [link, linkAddress]),
    filter(([, { id }]) => id === companyAAddress.id),
  ))
)

---

However, I will say that the ability to get the linked companies from a single request should be implemented by the server.

Answer 2

You should consider another layout here.

Option 1

move that business logic to the server side. From the description of your problem, I can not see that the client simplifies in any way things to retrieve B's address. The server side has all the information it needs to retrieve all linked addresses.

You would only need one call to the backend. And you would avoiding await here, which makes the haptic also slow for the user if the server does not respond quickly.

pseudocode on server side:

// this is a new endpoint on the backend which holds your current client logic
on(/api/companies/<company>/address_links) {

  // retrieve the address with it's id like your first endpoint and store it temporary in address/id
  address = getCompanyAddress(company); 

  // query all all addresses with that id/address and store it in links[]
  other_companies[] = getCompaniesWithAddress(address);

  // Build your response JSON
  return_me = {
    id: address.id,
    addressLine1: address.line1,
    links: other_companies 
  }

  // send the response
  response.send(return_me);
}

Option 2

Redesign the backend and speed up the queries with websockets. You also circumvent the limits of simultaneous HTTP requests (It's only one per service).

Websockets can be slower for single requests if the connection ha not already set up but has substantial smaller overhead than several HTTP requests. A very nice article (from an author of a very nice framework called FeathersJS) can be found here: https://blog.feathersjs.com/http-vs-websockets-a-performance-comparison-da2533f13a77

You change your component code to listen to an Observer

this.subscription = this.myService.addressObserver$.subscribe((data: any) => {
  [Assign and do something]
}

and change the service to use a socket (not complete and not runnable, only an example):

import * as feathers from '@feathersjs/client';
import * as io from 'socket.io-client';
[...]

@Injectable()
export class MyService {
  private readonly feathersService: any;
  public myObserver$: Subject<any>;

  constructor() {
    const socket = io(<url>, {
      transports: ['websocket'],
      forceNew: true,
    });

    const feathersApp = feathers().configure(feathers.socketio(socket));
    this.feathersService = feathersApp.service('api/address');

    // on (create, delete, update) methods work like realtime when something changed on the server
    this.feathersService.on('updated', (address) => this.onUpdated(address));
  }

  // single find, can be adapted to query by something
  public find(): void {

    // the actual query to the server 
    this.feathersService.find(address).then((addresses: any) => {
      this.myObserver$.next(addresses.data);
    });
  }

  ...

Option 3

do both