Looks interesting! Do you guys offer any sort of visibility tools/reports into customer usage of different endpoints, tracking of actual API requests (including payloads/etc) per request, etc? Is it possible to use you guys just for billing if we already have our own auth?
Hey Dylan! A bit more context on each of your questions:
With regard to endpoint visibility, yes, totally. One of our strengths is the ability to even price different endpoints differently. If you have endpoints that are high-value that you want to track differently than other endpoints, you can simply tag it, and Kable will monitor that endpoint as its own dimension. So yes, you can see which customers are calling which endpoints through our dashboard.
With regard to tracking request payloads, this is something we do not currently support. We've given this some thought, and while we might change our position in the future based on customer feedback, we are currently not storing customer request data. The reason for this is privacy. We want to be careful about what information we store from our customers' customers, so we require developers to explicitly define what information gets recorded in Kable.
Finally, with respect to billing and analytics and authentication as separate features or products: *ABSOLUTELY*. You can use Kable for billing and analytics without using Kable authentication. We want it to be super easy to use both, but we understand many customers have other systems for managing API keys or other parts of the stack. In this case, we recommend using Kable's `record` method, and using us for our core competencies of billing and analytics.
Agree. The places I go on the web have become more and more centralized/limited. I think projects like this that help to surface and aggregate interesting content from the web (which is really what I come to HN to find) are great.
Nice work! The UI is really simple - and love not having to log in to use it. Have you thought about leveraging the ListenNotes API (https://www.listennotes.com/api/) to automatically pull in the podcast episodes via search vs having to upload them?
I agree with all your points - but one thing I think about is: how do we fix what we have today? How do you fix the concrete jungles that most cities are today in the US. Or is it inevitable that more concrete will just be poured over time until some major natural disaster allows for a reset?
The Netherlands has reversed course from a car-centric culture to a more pedestrian- and bike-friendly culture. They did that very intentionally. As far as I can tell, there is no magic about it being The Netherlands -- except possibly the influence on the culture of the polder.
From what I gather, protecting the polder so everyone survives is so important that it has, at times, required people at war to cooperate in keeping the water out.
Today, globally, everyone knows we need solutions here or we are all doomed. We can stop quibbling about what various factions want for personal gain and start seeking answers that help some group live lighter on the land so we all benefit.
Rinse and repeat.
Start with low-hanging fruit. As that gets done, other things will become more reachable.
I think the Netherlands is still one of the more car-centric cultures in Europe though. And I've lived in several countries. Perhaps the inner cities aren't, but outside of them it's very hard to do without a car.
In many small villages/towns there is only a bus service during peak hours now. When I visit it's really a royal pain (and taxies are unaffordable as an alternative). And when you work in the Netherlands you're usually required to work nationwide which means countless hours in the car visiting clients. Public transport takes several times as long as car travel.
I really hated it when I worked there (I'm from there as you might have guessed). All these hours driving in frantic traffic were so stressful. I work in Barcelona now where public transport is much better (rural is still worse than inner-city but both are much better than the Netherlands' services). It's the first place I've lived where I genuinely don't need a car, it would only be a burden to me. Time between metros is counted in seconds and the network is so big. As well as that there's buses and trams and regional trains passing through that can be used to hop from one side of the city to the other.
The only thing that's genuinely better in the Netherlands is the bike lane network IMO :) That really is amazing. But I just don't see the feasibility of doing without a car completely there.
> I work in Barcelona now where public transport is much better...
Barcelona is one of the most stressful, if not the most stressful city, to use public transport in in Europe. If anything it should serve as an example for other cities as to what to not do. Public transports are full of thieves and scammers. It is simply beyond belief. It is known that if you ever dare to retaliate when you catch people stealing you, it can quickly degenerate very badly... For you.
I don't want to hear the typical: "If you look like a local and know what not to do, you'll be fine". I want public transports to be very safe otherwise I won't use them.
Several people mentioned Tokyo already: I spent close to a year there. Now that is a city with working and safe public transports.
Yes Barcelona has a big problem with pickpockets indeed. This is more of a legal issue: any theft of 400 euro or less is punishable by a fine only even if it's the 4000th time.
Because of this there's gangs of professional pickpockets. But they're trained to avoid conflict. Because any violence will incur serious charges. I've grabbed one once and pushed him against a wall when I felt him reaching for my phone and he literally was passive and relaxed. Just dropped the phone and strolled off. No way someone behaves like that unless they have trained it.
So yes they're an absolute plague but the risk of violence is low. And the cause does not have much to do with public transport (it happens on the street too) but with ignorant lawmakers :)
> Public transports are full of thieves and scammers
This is true in the overwhelming majority of public transport around the world.
> Several people mentioned Tokyo
Yes, Japan is one of the safest nations in this regard. That's cool, but definitely not achievable everywhere because it is due to a lot of environmental factors (like the culture and the incredible conviction rates).
I think your argument only strengthens mine. It sounds like you are saying "The Netherlands was kind of the America of Europe in terms of being crazy car-centric and they sucked so much worse than they do currently."
To which I say "So you're telling me if they can become the global poster child for doing this better, anyone can do this better -- even the US."
Well, I don't know how much they suck now as I've left 10 years ago, but I know pre-corona there were many cases of the largest traffic jams ever reported in the news, almost weekly :)
When I was there these numbers were much lower despite the A2 being a horrible ever-changing construction site. And it was already so frustrating I once broke my teeth just from grinding it :(
At the same time even the frequent train routes were apparently really overcrowded. Everything is different now with Corona of course but that's worldwide.
It also helps that the country is old enough to have developed before the automobile. Kinda hard to put the toothpaste back in the tube when you've designed entire cities around support the automobile.
>It also helps that the country is old enough to have developed before the automobile. Kinda hard to put the toothpaste back in the tube when you've designed entire cities around support the automobile.
I think this is a misconception for sure - the netherlands used to have much more car-oriented development but saw the issues with it and started retrofitting their built environments.
The same goes for the U.S in reverse, actually, with a lot of cities not having been designed with the car in mind and then subsequently having been demolished and retrofitted for the car. The transition is totally possible to do in both directions.
> how do we fix what we have today? How do you fix the concrete jungles that most cities are today in the US.
Elbow grease. Break the concrete and plant shit, pay people to maintain these things every once in a while. I mean most amenity planting is low maintenance and only needs looking after once or twice a year to avoid it growing wild.
The US has the solutions already - a lot of money, and a lot of people looking for a steady job. All it takes is for people to stop hoarding said money and Decide to solve the issue.
Somerville, Massachusetts had a large Italian population a generation or two back and it seems damn near all of them considered "making it" covering every square inch of their land with concrete. Then putting up a 2 foot high brick wall topped with a few-feet-tall wrought iron fence.
The city has been working to undo it by providing financial incentives for removal, and partnering with an informal group of volunteers to make it cheap to do. They put out a call for public volunteers to have a sort of reverse-barn-raising. A big bunch of people show up and help rip apart and load up the concrete.
Jamaica Plain, a neighborhood in Boston, used to have a group of volunteers that worked with local nurseries to plant a tree and take care of the initial critical care (watering regularly) for free. All you had to do was email them, and then point when the crew showed up.
This blog is more of an intro to a few high level concepts (multi-GPU and multi-node training, fp32 vs fp16, buying hardware and dedicated machines vs AWS/GCP, etc) for startups that are early into their deep learning journey, and that might need a nudge in the right direction.
If you're looking for a deep dive into the best GPUs to buy (cost/perf, etc), the link in the below comment gives a pretty good overview.
PS - I can send you some benchmarks we did that show (at least for us) Horovod is ~10% faster than DDP for multi-node training FWIW. Email is in my profile!
In general - this is expensive stuff. Training big, accurate models just requires a lot of compute, and there is a "barrier to entry" wrt costs, even if you're able to get those costs down. I think it's similar to startups not really being able to get into the aerospace industry unless they raise lots of funding (ie, Boom Supersonic).
Practically speaking though, for startups without funding, or access to cloud credits, my advice would be to just train the best model you can, with the compute resources you have available. Try to close your first customer with an "MVP" model. Even if your model is not good enough for most customers - you can close one, get some incremental revenue, and keep iterating.
When we first started (2017), I trained models that were ~1/10 the size of our current models on a few K80s in AWS. These models were much worse compared to our models today, but they helped us make incremental progress to get to where we are now.
Dylan from Assembly here. If you want to send me one of your audio files (my email is in my profile) I'd be happy to send you back the diarized results from our API.
You can also signup for a free account and test from the dashboard without having to write any code if that's easier.
Other than lots of crosstalk in your group conversations - is there anything else challenging about your audio (eg, distance from microphones, background noise, etc?)
Great question. This is technically referred to as "Wake Word Detection". You run a really small model locally that is just processing 500ms (for example) of audio at a time through a light weight CNN or RNN. The idea here is that it's just binary classification (vs actual speech recognition).
There are some open source libraries that make this relatively easy:
This avoids having to stream audio 24x7 to a cloud model which would be super expensive. This being said, I'm pretty sure what the Alexa does, for example, is send any positive wake word to a cloud model (that is bigger and more accurate) to verify the prediction of the local wake word detection model AFAIK.
Once you are positive you have a positive wake word detected - that's when you start streaming to an accurate cloud based transcription model like Assembly to minimize costs!
Users expose their model to our Trial API (https://docs.determined.ai/latest/topic-guides/model-definit...), the base class then implements a training loop (which can be enhanced with user-supplied callbacks, metrics, etc.) that has a whole bunch of bells and whistles. Easy distributed (multi-GPU and multi-node) training, automatic checkpointing, fault tolerance, etc.
Concretely, the system is regularly taking checkpoints (which include model weights and optimizer state) and so if the spots disappear (as they do), the system has enough information to resume from where things were last checkpointed when resources become available again.
This is tricky. The de facto metric to evaluate an ASR model is Word Error Rate (WER). But results can vary widely depending on the pre-processing that's done (or not done) to transcription text before calculating a WER.
For example if you take the WER of "I live in New York" and "i live in new york" the WER would be 60% because you're comparing a capitalized version vs an uncapitalized version.
This is why public WER results vary so widely.
We publish our own WER results and normalize the human and automatic transcription text as much as possible to get as close to "true" numbers as possible. But in reality, we see a lot of people comparing ASR services simply by doing diffs of transcripts.
