Swiss deals API

Picture: Gaël G.

Introduction

A friend of mine told me how he missed out on a fantastic offer on a gaming monitor he really wanted. Then, he wondered if there was a platform that collected the deals of the day on the online stores that do it. We can mention the website piratesdesprix.ch which lists different deals from several shops. However, they did not have the stores we were interested in, namely QoQa, Digitec and Galaxus. It is from here that I had the idea to create a small service that aggregates the deals of the day!

gaelgoth/swiss-deals-api

Aggregate deals of the day from Digitec, Galaxus, QoQa

Building API with Golang

Used to create backend in Python or TypeScript/JavaScript, this is a good opportunity for me to try Golang. Before we start coding the API, we need to figure out how to get the information we need. Then we will set up the Go dev environnement. Finally, we will be able to retrieve data from QoQa, Digitec and Galaxus.

Retrieve data

Let’s do some reverse engineering to find the best way to get the information about the offer of the day.

Let’s get started with QoQa. By analyzing the traffic when we load the page, we can easily notice the http request that serves us the promotion of the day. Get http deal from QoQa

Basically, we found out that the following Request URL give use the deals: https://api.qoqa.ch/v2/websites/wwwqoqach/offer_preview. The wwwqoqach identifier can be retrieved from the following endpoint: https://api.qoqa.ch/v2/websites

bash
# get identifiers list
curl https://api.qoqa.ch/v2/websites | jq
# get current deals by replacing <identifier>
curl https://api.qoqa.ch/v2/websites/<identifier>/offer_preview

Unfortunately for Digitec and co, this is a different matter. The offer of the day is displayed via several GraphQL queries. I am not very familiar with GraphQL, so for Digitec the idea is to scrape the page that gives us the offer of the day.

Golang frameworks

In a nutshell, we require web frameworks that support HTTP methods and HTML scraping in order to build this API. Swiss Deals API will based on Fiber that is an Express inspired web framework written in Go. We’ll utilize Coly, an elegant scraper and crawler framework for Golang, to extract web site content. To power this API other sub-tools are exploited for swagger documentation, query caching and logging.

How Swiss deals will work

Environnement

The development environment with Golang is relatively easy to set up. I tested several alternatives:

Makefile: The startup task runs the command go run main.go and writes its process ID to PID_FILE.
Docker-compose.dev: Run the Golang build inside a container coupled with Air for hot reloading when editing the code from IDE
Air (the easiest one): As described by his creator, just air command in your project root directory, leave it alone, and focus on your code.

Everything is ready, let’s see what Swiss Deals API can accomplish after a few hours of code.

Demo

At the time of writing, the API serves two groups of endpoints, “QoQa” and “Digitec”.

bash
$ curl https://deals-api.gothuey.dev/api/deals/digitec | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   389  100   389    0     0    805      0 --:--:-- --:--:-- --:--:--   805
{
  "store": "Digitec",
  "title": "Netgear RBK353 Orbi WiFi 6 Dual Band set de 3",
  "url": "https://www.digitec.ch/liveshopping/",
  "remaining_stock": 61,
  "stock": 150,
  "offer_price": 179,
  "regular_price": 267,
  "image_url": "https://www.digitec.ch/im/files/..."
}

You can interact with Swiss Deals API by consulting the Swagger documentation on https://deals-api.gothuey.dev/swagger/index.html.

Final thoughts

I was able to learn Golang programming through this short project. Along with improving the quality of the code, my goal is to increase the variety of offers the API returns.