Questions tagged [data-integrity]
The data-integrity tag has no summary.
24 questions
1
vote
2
answers
138
views
Designing persistence guarantees in an ingestion pipeline with a non-customizable intermediary
I’m migrating a large volume of historical RRD data into a VictoriaMetrics cluster.
By design, vminsert buffers and asynchronously forwards data to vmstorage. This makes it unclear when data is ...
- 21
3
votes
1
answer
127
views
Constraint to enforce pairwise distinctness of values in two columns in table
I'm kind of stumped by a seemingly simple problem in Db design, but the more I think about about it, the more it twists my mind.
Let's say I have columns C1 and C2 in a table. How do I enforce that ...
- 141
-2
votes
3
answers
273
views
How to present serious flaws in third-party software to non-technical users
We have third party software being tested. Our software has to interface with this, hence my involvement. There are some things I consider serious technical flaws. I'd like to communicate these things ...
- 301
0
votes
2
answers
148
views
When reconstructing file B from file A and a binary difference B-A, given that A and B-A are correct, do we still need to verify the integrity of B?
In the context of a software update system, suppose we have the following:
a.tar: an archive file containing an old version of an application
b-a.patch: a file containing the binary difference of the ...
- 129
0
votes
3
answers
276
views
In data engineering, why is data integrity checked on the DW rather than on the data sources?
I'm a software developer and new to data engineering, so this may be a newbie question, but I'm wondering why data integrity checks (for instance, dbt tests) are ran on the data warehouse, rather than ...
- 243
0
votes
2
answers
204
views
How to verify that a legitimate (but unknown) remote asset from an unknown source has not been compromised and that its integrity remains intact?
I am familiar with and see the benefits of Subresource Integrity (SRI).
I understand that with SRI, once you've added a script reference with the correct integrity attribute, if the remote script is ...
- 285
-2
votes
1
answer
438
views
DDD Best way to preserve integrity of entity
I have an entity that contain the status of data (WaitingForData, ReadyForData, DataUsed).
class Batch
{
int Status { get; }
DateTime StartTime { get; }
string Code { get; }
void ...
- 67
1
vote
0
answers
58
views
Best way to store webshop item data as a snapshot when an order is created
I'm designing the architecture of my application and I'm running into an issue. For the webshop every Order entity has a relationship with a Store and many StoreItem's. This allows for quick fetching ...
- 19
11
votes
6
answers
4k
views
Is CRC pointless if I'm doing truncated HMAC?
I am currently designing a communication protocol for an embedded system. I've decided that authentication (but not encryption) is important, so I decided to add 4 bytes of truncated HMAC signature to ...
- 2,099
-3
votes
1
answer
110
views
what is event based data integration? [closed]
Please help me to understand what is event based data integration in simple layman term with some examples?
How it is different from other form of data integration.
Some sample use cases will be ...
5
votes
5
answers
2k
views
Relational integrity without relations
I have about 15 years software engineering experience, writing business software with relational databases. Mainly SQL Server and Oracle. I've always been of the opinion that you should define ...
- 301
3
votes
1
answer
117
views
Is it possible to spoof a recipient in a SMTP transaction? If not how does SMTP prevent this from happening?
Is it possible to have a scenario where there are 3 email addresses. Each on a different mail server.
[email protected]
[email protected]
[email protected]
Charlie writes an email with Alice and ...
- 141
0
votes
1
answer
210
views
Design/Process in respect to mission-critical systems (Web Services)
Overview
For company abc, teams are separated in terms of mission-critical applications and their extended applications. For example, customer xyz uses mission-critical data, but there are business ...
- 414
2
votes
1
answer
167
views
User identity between two services
Our team has hired freelancers for creating a website.
They did their job well, but now we want to add services for our core
product using different server and technologies, but we want user
...
- 259
12
votes
2
answers
401
views
How do you design your software that updates several microservices, if one of them fails?
Is there a design pattern or practice I can use to help with services that are either down or go down, while others are stable?
What if I have three microservices, and two of them are good, and one ...
- 3,679