> The CAP theorem is also irrelevant if your acceptable response time is greater than the time it takes your partitions to sync.
This is really an oversimplification. The more important metric here is the delay between write and read of the same data. Even in that case if when the system write load is unpredictable it will definitely lead to high variance in replication lag. The number of times I had to deal with a race condition for not considering replication lag factor is more than I would like to admit.
We have been using Cloudflare health check at work for this and we have been pretty happy with this. We have this integrated with slack and opsgenie as well.
The piVPN story is interesting. I am currently stationed outside my home country. Using the financial apps (bank apps, wallet apps) outside the country are restricted hence I needed a ip within the country. No major VPN provider provide IPs for my country. piVPN has been great in that sense to access the contents from my country.
I think BullMQ with deployment in kubernetes gets people off the ground quite quickly and scale out quite well without needing to worry about fine tuning the solution. Also RabbitMQ has also worked pretty well for us handling around 150k RPM messages with delayed exchange enabled while running in kubernetes without any tweaks.
As a dev working on solution that is used in the industry of customer support, I am more worried about,
1. Data locality
2. Privacy
3. SLAs and Uptimes
While SaaS solutions like these can get us MVP/PoCs fast but a home grown solution is more preferable when SLOs are tight and security is a huge concern.
I agree with you, as mentioned in the article, I’ve built similar solutions as yours at Algolia, using Redis and Kubernetes.
However, not all developers know and want to do Kubernetes or RabbitMQ and not all companies can afford to invest time to set up and manage it (https://docs.bullmq.io/guide/going-to-production).
We address privacy by encrypting all the data on our side (doing a second pass with a symmetric PGP key for tokens such as GH tokens, and environment variables) and advise companies that want to keep their data on their infra to push as minimum data in arguments while leveraging a dedicated SSH tunneling setup between our infra and theirs.
When it comes to the SLA/up-time of home grown, my POV would be that, again, achieving good results on those often requires SRE engineers, which is an investment.
yellow.ai | Frontend Engineer, Backend Engineer, Fullstack engineer at all levels | Bangalore, Dhaka, Remote (Around Indian Timezone) | https://yellow.ai
South East Asia's leading Artificial Intelligence powered Voice and Messaging platform for enterprises. As the customer engagement is moving towards chat and voice, enterprises are rapidly adopting YellowAI to enable them to engage with their customers with enterprise virtual assistants.
We are looking for passionate engineers who can help us build the vastly growing product. We use a microservice based architecture powered by Kubernetes.
Yellow Messenger | Full Time | Frontend Engineer, Fullstack Engineer | Bangalore, Dhaka, REMOTE (around IST timezone)
Yellow Messenger is a Bangalore based Conversational Engagement Platform. We are a tightly knit engineering team and growing. We have recently raised our series B investment of $20M from Lightspeed Ventures. At Yellow Messenger, we help enterprises build chatbots that help them grow their engagement across all messaging channels. We have a proven solution that works for all industries. We are currently processing over 1B messages per month and growing. Currently we are looking for passionate Frontend (React & Redux) and Fullstack Engineers (NodeJS) who can be a part of our small clan.
What the team uses currently - Language: NodeJS - Frontend: React, Redux - Databases: MongoDB, MySQL, Redis - Infrastructure: Kubernetes - Analytics: Druid - Other tools: Kafka, Jenkins
Requirements: Minimum 2+ Years of experience
If you are interested please mail me at anik@yellowmessenger.com and add [HNHIRING] to the subject.
Yellow Messenger | Full Time | Frontend Engineer, Fullstack Engineer | Bangalore, Dhaka, REMOTE (around IST timezone)
Yellow Messenger is a Bangalore based Conversational Engagement Platform. We are a tightly knit engineering team and growing. We have recently raised our series B investment of $20M from Lightspeed Ventures. At Yellow Messenger, we help enterprises build chatbots that help them grow their engagement across all messaging channels. We have a proven solution that works for all industries. We are currently processing over 1B messages per month and growing. Currently we are looking for passionate Frontend (React & Redux) and Fullstack Engineers (NodeJS) who can be a part of our small clan.
What the team uses currently - Language: NodeJS - Frontend: React, Redux - Databases: MongoDB, MySQL, Redis - Infrastructure: Kubernetes - Analytics: Druid - Other tools: Kafka, Jenkins
Requirements: Minimum 2+ Years of experience
If you are interested please mail me at anik@yellowmessenger.com and add [HNHIRING] to the subject.
