We would all prefer to have access to a public API but sometimes, services you would love to add automation to only offer their website to use.
There is no public API, now what?
First let’s state that not all mobile or web applications can be (easily) reversed engineered, some companies invest a lot of money to secure and protect their API, user data. I’m not a hacker, but sometimes being resourceful let you create scripts and automations that can make you save tons of time. Note that private API, as in not documented, are usually not intended to be used by others than developers internal to the service or product, so don’t count on it to be stable, don’t expect to receive a notice if something is going to break, don’t base your entire business on something that can disappear any day.
In some cases, web browser inspectors comes handy. With the surge nowadays of websites developed using React, or more generally client side web applications, the network inspector of your browser often shows all the requests the website is doing in the background in response to your actions on the interface. Coupled with a HTTP proxy like Charles to inspect traffic going through your phone, you can scan pretty much all endpoints a service is using and try to mimic this behavior in scripts.
What’s the catch?
There is often only one problem: authentication. Most web services nowadays are using JWT session tokens, passed in the headers of each authenticated request, and even though you could re-execute a request you copied from your network inspector / HTTP proxy using the same URL, parameters and headers, this session token will eventually expire after an hour (a day at best depending on how their authentication is configured) and you will have to manually extract your access token again and again.
To be honest, I’m often able to reverse engineer everything but the authentication, as a user, I’m fine with that š
Refreshing tokens
This is the best case scenario: if you can sniff and extract the refresh token from the original authentication request, and reverse engineer the token retrieval request, then you can generate a fresh access token every time your script run.
Let the website refresh tokens for you
Chances are the website is refreshing its own access token periodically as well as when launching it, this provides a seamless experience without having to log in every time. Obviously websites trying to secure sensitive data like banks are not doing this, but a lot of other consumer websites and tools are.
In that case, it’s pretty common for these web applications to store access tokens and / or refresh tokens in their local storage. You can see the content of the local storage using the “application” tab of the web inspector (Google Chrome), sometimes this information can also be stored in cookies.
Final note
Don’t be evil.
Remember that you can be in trouble for doing that, only use these techniques to automate something you have the right to do.