Gophers like waiting in Postgres queue

As we as humans must stay in the queues for bananas, the Gophers should sometimes stay organised too. You may use various technologies for simple queueing, but you can also easily use Postgres too. Write the reliable consumers in Go on top of Postgres tables and keep your architecture simple...

Place
Golab #1
Length
60 min
When
November 21st, 2023
11:00

Abstract

Queues

Plenty of go projects in their lifetime come to the point when they need to involve asynchronicity when processing some workload. Usually a good way to implement the asynchronous behaviour is to involve some message broker (queue mechanism). There are various technologies the Go developers may use (Redis, RabbitMQ, Kafka, SNS and many others). However, when you just need to use the basics queue mechanism to help you better distribute the jobs for Gophers (Go services replicas) these tools are usually too big gun to be used.

Our story in Dataddo.com

In dataddo.com we struggled with this a bit. We tried using some technologies mentioned above, but we faced the reality with too complex configuration problems, or the abandoned libraries or packages that where limiting us too much. Also we did not want to make our architecture complicated too much and wanted to use the technologies we were already using. So we decided to implement our custom queueing mechanism written in Go using the Postgres database. In our concrete case of switching from RabbitMQ to Postgres we made our developers happier as the used technology stack did shrink and we got nice observability about the "to be processed", "being processing" and "already processed" jobs in queue as a gift.

Stay connected

You can subscribe to our newsletter to stay updated on the latest news and announcements. Follow us on social media to join our community and engage with other attendees, speakers, and staff. We're on Twitter, Mastodon, Youtube and Facebook. Don't forget to use our official hashtag #GoLab2023 to share your experience and connect with other attendees on social media.

Subscribe to our newsletter

We hate spam just as much as you do, which is why we promise to only send you relevant communications. We respect your privacy and will never share your information with third parties.
©2023 GoLab | The international conference on Go in Florence-Design & devCantiere Creativo-Made withDatoCMS