Timeline for What are the different approaches to filtering on aggregated data?
Current License: CC BY-SA 3.0
19 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 29, 2016 at 23:57 | comment | added | Ewan | im not suggesting you use a cdn | |
| Nov 29, 2016 at 23:56 | comment | added | Andy Baird | Just needs to be real time as of the time of loading the page or applying a filter. I see now that's what you meant in your previous comment. A CDN with a timeout still would end up with cache hits when it shouldn't, but that's just one of the myriad of reasons of why I'm not going to put my data on a CDN. Anyways, thanks, I have a solution figured out at this point. | |
| Nov 29, 2016 at 23:49 | comment | added | Ewan | you'll need to use websockets or similar to constantly stream the db updates to the clients. switching to event sourcing is probably advisable. | |
| Nov 29, 2016 at 23:46 | comment | added | Andy Baird | How so? It's not bigger problems, it's just different problems. | |
| Nov 29, 2016 at 23:43 | comment | added | Ewan | if you really need the changes between loading the page and clicking sort by x you have bigger problems | |
| Nov 29, 2016 at 23:35 | comment | added | Andy Baird | Real time. So the timeout would be zero. So, it wouldn't effectively cache anything. | |
| Nov 29, 2016 at 23:31 | comment | added | Ewan | obviously in your situation you would simply cache the data with an appropraite timeout. you still gain the benefits | |
| Nov 29, 2016 at 23:14 | comment | added | Andy Baird | Not really - as previously mentioned this needs to be real time and the data changes pretty often, never mind the security nightmare that would be putting our company's data on a CDN endpoint. Definitely not a good solution for my problem, but thanks. | |
| Nov 29, 2016 at 22:58 | comment | added | Ewan | yeah json blobs. tbh it seems analogous to your problem. 'scoped to company' == 'scoped to womens wear' the key problem these companies have is the volume of users and the bottle neck of the database. (most run on old relational ones) as the data is public and changes rarely a CDN is a good option. | |
| Nov 29, 2016 at 22:52 | comment | added | Andy Baird | We're quite off topic at this point, but I'm very curious how and why an ecommerce site would use a CDN as a datastore (and what would it even cache as? JSON blobs?) and not something more tailored to that sort of operation (like elasticsearch). Do you have an example ecommerce site that does this? | |
| Nov 29, 2016 at 22:47 | comment | added | Ewan | its a common approach for e-commerce sites for example where you have say a bunch of clothes in various styles, colours, sizes etc and have many users browsing. you drastically reduce the load om your back end servers and the user gets a responsive experience as they change the filters and sorting | |
| Nov 29, 2016 at 22:46 | comment | added | Andy Baird | That would only be true if they actually used every single filtering option on a single visit. Definitely not the case for this. The data is segregated by company and not identical for all users (as I stated in the question)... besides, putting your data store behind a CDN? Seriously? | |
| Nov 29, 2016 at 22:41 | comment | added | Ewan | the total data download is less unless your users never look at more than the first page and dont change the filtering/ordering more often than the page count. Also because the page load is identical for all users it can be cached and put on a CDN. Once you offer the user more than 2 or 3 filtering options the UX is far superior to the reload on every click option | |
| Nov 29, 2016 at 22:32 | comment | added | Andy Baird | Yeah, that's definitely not a good idea in my case. I can't think of any other website that would download force a user to download megabytes of data on page load just to accomplish client side filtering and sorting. That sounds like a nightmare for the end user. | |
| Nov 29, 2016 at 7:38 | comment | added | Ewan | yes download all data. | |
| Nov 29, 2016 at 2:49 | comment | added | Andy Baird | Sorting and filtering on the client side is standard behavior? Maybe on small datasets, but I don't think that's the norm for large sets of paginated data. I also don't see how you'd ever send less data unless you're talking about filtering on only 1 page worth of data at a time, otherwise you'd have to download all pages of data at once. | |
| Nov 21, 2016 at 8:26 | comment | added | Ewan | its pretty standard behaviour for websites these days. If the user changes these options on the fly then you will actually send less data this way | |
| Nov 21, 2016 at 6:50 | comment | added | Andy Baird | I really can't imagine sorting and filtering on the client side - that seems like an objectively bad idea given the potential volume of the data set. Even keeping it in the server side code seems a bit foolish - databases are designed to efficiently sort and filter on your data. I can think of some edge cases where that might be a good idea, but not many where it should be your go-to option. | |
| Nov 18, 2016 at 9:08 | history | answered | Ewan | CC BY-SA 3.0 |