Does anyone else get intermittent 503 "upstream error or disconnect" when running istio? We get this fairly regularly and have to patch services to get it to run again. Would love to share ideas if anyone faces similar issues.
Nope, not a junior. For our staff roles, we have Junior - Mid Tier - Senior - Principal. This is for a Mid Tier role.
It's just how it is here, sadly - I wish we could change it, and so does my boss.
Nobody is here for the money; we get to work on pretty cool projects with quite a big user base (we're talking millions of users and billions of request per day) - No. 6 Alexa rank for the U.K, and one of 2 non-startups in the Top 20. We have a lot of authority and a relatively relaxed work environment
You have one thing right. It is very Sad. You will either hire dreadful developers or junior developers, Salary is FAR too low BBC!! Dont be Cheap now!
So I think in this instance I would deem valid to be:
1) the url is the correct format
2) the url resolves to an ip address
3) the url is registered and is in use. By this I mean it's not one of the "this domain name is for sale" pages.
Number 3 is the novel and challenging piece of this.
It works by using the same private APIs that the bank's own mobile app. We reverse engineer their app, work out the API contract, implement our client, and normalize the data.
Reverse engineering mobile APIs is a superior strategy to screen-scraping because:
- they already return structured data
- the security model for that channel is different, e.g. no need for 2FA all the time so truly unattended use cases are possible
- it's much more difficult for banks to make breaking changes. When banks do make a breaking change the old version is supported for a decent amount of time to allow their own banking app users to upgrade, we can take advantage of this window to provide a consistent level of service.
Finally, we often don't need user credentials. If there is an option to authenticate with another mechanism during registration, e.g. EMV CAP (A.K.A Barclays PINSentry etc) we use that.
Not only does this sound potentially illegal but how can you be confident that you will recognize the breaking changes in time to fix them? What if you begin supporting a large number of banks and you can't keep up?
Also, will your reverse-engineered use of the mobile API's have any detrimental effect on the user? I imagine the user will be the one authenticated with the API, what if the bank starts to see an influx of odd API traffic and decides to investigate it or there is some type of rate limit?
Your decision to reverse-engineer mobile API's opens the door for many important questions in my opinion.
The EU Computer Programs Directive 2009 provides an exemption for reverse-engineering for the purposes of creating inter-operable systems. This directive has been harmonized into UK law (where Teller is domiciled and operates) and Teller satisfies the requirements to be protected by the exemption. We have also developed many novel techniques that do not meet the UK legal definition of reverse-engineering so we have that angle too. This issue has also been looked at by expensive lawyers.
In terms of stability. It actually takes 6-12 months for a bank to get something into production. We are not talking about fast moving organisations here. We have not had a breakage with a supported integration in two years of beta testing.
We take many steps to ensure our traffic does not stand out to banks eager to actively interfere with Teller. Our clients perfectly emulate (100% API compatibility with their own) and make the same API calls in the same order etc. We also only make API calls as a result of user action, i.e. Teller does not poll or cause atypical traffic patterns. Finally have 100s of IP addresses and assign an IP address to a user for a period of time. All of this compounds to make Teller traffic look indistinguishable from their own mobile app traffic. The objective is to make it more likely they will block their own app traffic than block Teller as a string incentive to not interfere their customers' choice to use Teller enabled services.
Hey Stevie, I was at the HN London where you gave a very memorable demo on reverse-engineering mobile banking apps. Stoked to see to how far you've come and congratulations on the Teller beta launch!
Even back then you had caught the attention of banks. I'm sure they've threatened you many times. But now that banks are taking you more seriously and returning your calls, how are you going to convince them to work with you instead of against you?
And what happens when, if they haven't begun already, try and legally DoS you?
I hope banks will realise that open APIs are a good thing, and if they don't start getting their shit together, they'll be left behind. Our whole financial infrastructure is so needlessly complicated. Why can't it all be JSON APIs?
I've used similar to scrape real estate data. Reverse engineering mobile APIs is often the most consistent way to get programmatic data access these days. Just grabbing a bit of data as a user this isn't a big deal - but I wouldn't want to base my business around it, it seems like there are too many opportunities for that to blow up in your face.
What happens if the banks realize you're doing this and just block your IPs? What happens if they decide to go after you legally because you reverse engineered their applications and are benefiting from it commercially - you'd have a decent argument, but are you prepared to go to court with it?
Then again I guess the screen scrapers have gotten away with it for a fairly long amount of time, so maybe it's not a concern...
Sure they might find out an App Update was issued in the App Store... but they cannot possibly know what API changes were made without reverse-engineering the API's all over again.
Not to mention, once they discover what changed, now this service has to go make the changes on their system... meanwhile that bank doesn't work with this service anymore.
So the App is broken for some undefined period of time.
The bank has to deploy their client code to all of their users before they can depreciate the old API. That should give them plenty of buffer time to reverse engineer the changes.
"All" of their client code is a bunch of apps, that mostly auto-update. My bank (small town credit union) has a banking app that refuses to work at all unless it's the most recent version of the App. I see no reason why this would be a challenge for the bank - it's only a challenge for this service.
That doesn't stop them from deploying new APIs to their client software deactivated. Then "flip" a switch to cause it to behave differently all at once.
Sure, but they've got plenty of buffer time since their app doesn't break when the new version is released, it breaks when the old API is actually depreciated. Just because you release a new version on the App Store doesn't make clients with old versions of the app disappear.
Consider that e.g. Barclays randomly breaks their own mobile app for substantial subsets of their own customers. Over the last 4 years or so, the app has been broken for me more than 2 of them because they apparently consider it acceptable to whitelist phones based on their own whim (if you have one of the big brand phones, presumably it usually works, but e.g. even users of relatively well known brands like Huawei can often find their new phones won't work with Barclays for several months before it gets whitelisted).
In other words, many of the big banks have such a dysfunctional relationship to internet banking that customers of many of them have learned to deal with not being able to use the apps for long periods of time - it's hard to imagine that a third party will be perceptibly worse.
Apps won't be updated instantaneously on devices, though. There'll have to be some backwards compatibility for a good while if the banks want to change their APIs drastically. You can't just shut down access to your customers apps.
> You can't just shut down access to your customers apps
Yes you can, and banks already do this. Even the E-Trade app refuses to work unless it's on the most current version. When you control the apps and the API, you can pretty much do as you please.
It's takes banks 6-12 months to get anything into production. Also breaking APIs to thwart me has to compete with delivering new features to users. When they do change the contract, we know about it. We watch for new app releases, and proactively check.
How do you guarantee SLA uptime with that model? You won't know of any breaking changes until it hits production and when it does you will have to fix it. This may be an easy 30 minute fix and you pause transactions, or it may be a 24 hour fix and your world is broken for "forever" in api standards where 99.9X% is generally standard uptime requirements.
It seems to be that any service that consumes private APIs is going to be inherently unstable, although I agree on the slow pace of work in the finance world.
Also, once they figure out your consuming their APIs there will be an arms race and possibly legal action.
I'm not sure you need to guarantee an SLA for this to be attractive. E.g. I've badly wanted to get automated access to my statements for ages, but not had the time to work out an approach. I'm used to my bank (Barclays) being so insanely backwards, that if Teller's API suddenly tells me "sorry, broken because your bank are morons" and the API is down for a couple of days while they work around it, I'll blame my bank, not Teller.
For comparison: I've been unable to use Barclays own mobile app for about half of the last 4 years in chunks because their app seems to whitelist phones as they decide it's worth it. On a relatively random subset of my phones over the last 4 years, it just shuts down. No error, no nothing. Their support insists nothing is wrong. Nobody at Barclays appears to give a shit that they lock customers out. The Play store is full of thousands of one star reviews from people affected. My current phone took 6 months before it suddenly started working.
In other words: If you bank with Barclays, either you accept that you'll need to have a pinsentry device on standby when you get a new phone, or you'll stand a good chance of being unable to use their service for months on end.
When I signed up for Xero for my business account, I had to provide them with a document that they had to fax to Barclays to get them to pass on transaction info. It took about 10 days to get it enabled. If only I could get my personal statements electronically as easily (maybe Teller can finally solve that).
The banks have made sure peoples expectations of them is so low that any third party service that's remotely transparent is likely to find customers are very tolerant of bank problems.
I mean, the only reason I'm still with Barclays given the above is that most of the alternatives are shitty too, and I have locked in a mortgage rate with them that I "can't afford" to give up as long as the Bank of England rate is as low as it is (tracker rate that means I pay well below inflation).
It uses the internal APIs mobile banks use afiak. For what it's worth the open API laws aren't in power yet in Europe and tbh I imagine nearly all banks will fail to implement by the deadline. Or if they do, the APIs will be fairly crippled.
So does Europe. Open Banking Ltd in the UK is attempting to develop these standards, but they are working with tens of stakeholders across impossible deadlines - which means they cut scope and quality. And since banks in the UK are sponsoring the show, it is beyond certain that no disruptive moves will be taken, and interests of the incumbents will remain preserved.
Many do, but the key is that the mobile apps generally only require access to 2FA for registration, so if they implement the mobile app interface they can get away with having users use the 2FA device once.
This is the case for Barclays, for example, where in fact once the mobile app is registered, that can be used as a Pinsentry (2FA) device replacement.
I'm in no position to answer authoritatively, but for what its worth the three accounts I have left in the UK, with different banks, can all be accessed without a 2 factor device.
Specific to development- I am a big fan of pluralsight. I have done hours and hours in courses on various topics and I think it's the best way to get going on something new for me.
