Post

Conversation

Every software engineer should read Google's Spanner paper. Spanner proved CAP theorem wrong by inventing TrueTime that achieved <7ms clock synchronization at planet-scale. Very few innovations impact software we use as much. Must read.
Image
David Watson 🥑
Post your reply

Stop wasting time reading 50 page documents. This AI tool is like ChatGPT for reading faster. Just upload any PDF and ask a question. It will give you an answer with page citations in seconds. Try it for free.
Application of Time API/systems and the advanced physics and technology behind them in software systems are so mind-blowing and impactful (billions of dollars saved on infrastructure cost each year for the big tech, not considering their bigger impact on new opportunities they
Show more
Not at all! Have you heard of hotspots in spanner? Claiming spanner disproved CAP theorem may not be right. Its still a triangle with CAP vertices. You may choose a side but can only have 2 vertices. In spanner the triangles angles are like 1,1,178. I hope i made sense.
It didn’t break the CAP theorem. In summary it riefied the concept and real time measurement of network delays to be included in protocol design. It enabled building optimistically responsive (improve as network bw improves) database spanner. Unfortunately one cannot
Show more
I wouldn’t say it proved CAP theorem wrong. It was more about showing with a perfect clock(which was very costly at that time), you wouldn’t need to make compromises :)
What if understanding innovations like Google’s Spanner and TrueTime isn’t just about advancing technical knowledge, but about rethinking the very limits of what’s possible in software engineering—how could this transform the way we build and imagine systems at a global scale?
> Proved CAP theorem wrong They didn’t. They did some brilliant work and had some very creative solutions. Absolutely worth reading this paper. But the CAP theorem has not been proven wrong so far.
Saying Spanner has solved CAP theorem is the tech influencer equivalent of pop science YouTubers trying to explain the double slit experiment (“omg how can it be both a wave and particle???? Unexplained!”) or something. You have no idea what you’re talking about.
How does it solve CAP theorem? What happens to a cluster of nodes that are physically disconnected that need to write data that conflicts with the other branch?
It's also important to be familiar enough with Paxos that knowing the order of operations without needing to contact a leader means everything is faster.
I once published a paper with 5 co-authors (because two teams in the lab concurrently reached the results over a summer, and this was way to resolve the conflict). Even five was seen as too many! Good to know about this clock synchronization.
A theorem can never be disproved if the axioms are still same for the same system. A theory / hypothesis / conjecture / law / principle can be disproved, but not a theorem. As theorem itself has the mathematical proof associated with it for a set of axioms.
AWS has a competitor now, Aurora DSQL. More familiarity with that. Have to compare the two at some point. New system design answer: just use this. It’s awesome.
Spanner is 🔥. But I dont think it’s right to say that Google proved CAP theorem wrong since CAP, which I have alway interpreted a la FLP deals with asynchronous systems. TrueTime converts the system to a synchronous system instead.
Migrating to WarpStream to reduce your Kafka costs by 80+% and simplify operations is easy with byte-for-byte replication provided by Orbit. Use Orbit for migration, disaster recovery, cost-effective read replicas, performant tiered storage, low-latency geo replication and more.
Since its google, it’s likely over engineered, the code base is a disaster, the team moved to other projects and they will abandon it in 5 years or so.
Google is behind some of the greatest technologies we see today. It appears their goof ups are just tactics to keep the competitors busy for nothing
Breakdown of the paper: Spanner is a scalable, globally-distributed, and synchronously-replicated database system developed by Google. It is designed to handle large amounts of data across multiple datacenters while providing strong consistency guarantees. Spanner provides
Show more
Image
Show additional replies, including those that may contain offensive content