I've built a small scraper for German apartment listings (prices, locations, size). I originally started out wanting to get a bit deeper into Django's templating system as well as component libraries and needing some data to build on. Hence I ended up scraping apartment listings which in and of itself is pretty interesting data. I ended up writing this blog post about it, maybe it's insightful for some of you!
I think a general problem with most of these problems is that they’re biased towards how big US tech companies recruit.
However non-tech companies also want to assess coding skills but don’t care about algorithms & data structures.
A bunch of programmers today are not software engineers but analysts, data scientists, data engineers. So it might be worthwhile to examine how to access these applicants‘ skills.
For example for DS roles more Kaggle like real life problems might be suited better.
I’m just using Ibis: https://ibis-project.org/
They provide a nice backend agnostic API. For most backends it will just compile to SQL and act as a query builder. SQL basically has solved the problem of providing a declarative data transformation syntax, why reinvent the wheel?
It may also look a bit boilerplate heavy but with the VS Code add in and code snippets it’s actually quite ergonomic to write. Just don’t attempt to type it all out by hand.
If you decide to give it a try there are some packages I’d recommend:
- State management: https://riverpod.dev/ is easy enough and quite powerful. The official docs still recommend Provider, but that’s basically deprecated. Riverpod is an improved Provider by the same maintainer
The setup may of your dev environment may take a bit (the mobile SDKs are quite big to download) but after that it’s quite smooth.
Flutter is “inspired” by react so you should feel familiar with the widget lifecycle. Unlike modern react though it uses class based components and when using stateful widgets (components) it’s split into two separate widgets.
I like Plotly’s interactivity, but the Python API would be so much better if it was typed. I need to google which attributes to change to get anything done all the time. Copilot helps a bit but also constantly hallucinates plausible but not implemented plotly settings.
I also failed when trying to create a nice violin plot as implemented with matplotlib in the Shap library.
That said I tried using other charting libraries but Plotly’s interactivity is a killer feature others don’t offer in the same way.
While I think auth is hard it is still doable without having to become an expert when it comes to the details. I recently played around with oauth2proxy and nginx and got it working: https://github.com/layandreas/oauth-proxy-example
It indeed isn't that hard to get something up and running with oidc or oauth.
However, what becomes problematic is handling inherent complexity of whatever the provider you use is. If the OIDC flow is only intended for you or in-house at a company then it is less problematic. But when used by all sorts of people and devices, it will break in all sorts of problematic ways.
But that isn't to say that you shouldn't use the technology it is fantastic at what it does. I.e. handle social identities in the case of oidc, and authorization in the case of oauth