Hi! Author here. I agree that I should have explicitly stated the word "priority queues" since it is an ADT people can directly relate to. I will add it in. However, it is simply not true that I did not describe how a priority queue-based solution works.
I have described it in the "Timer Modules" section:
> A natural iteration of this approach is to store timers in an ordered list (also known as timer queues). In this scheme, instead of storing the time interval, an absolute timestamp is stored. The timer identifier and its corresponding timestamp that expires the earliest is stored at the head of the queue. Similarly, the second earliest timer is stored after the earliest, and so on, in ascending order. After every unit, only the head of the queue is compared with the current timestamp. If the timer has expired, we dequeue the list and compare the next element. We repeat this until all the expired timers have been dequeued, and we run their expiry processing routines.
And then, I go on to talk about its runtime.
Truth be told, this is a chapter for my book on data structures and algorithms that I think are interesting and obscure enough that not many people talk about them. Its goal is not widespread practicality, but rather a fun deep dive into some topics.
When my friends and I were undergrads (3rd year, I believe), we had an absolute blast exploring this exact topic - the intersection of Bloom Filters and client side searching. So much so that it became part of our undergrad thesis.
It all started when Stavros's blog was circulated on Hacker News! The way we approached the search part was by using "Spectral Bloom Filters" - https://theory.stanford.edu/~matias/papers/sbf-sigmod-03.pdf - which is based on a paper by Saar Cohen and Yossi Matias from the early 2000s - its basically an iteration on the counting bloom filters. We used the minimal selection and minimal increase algorithm from the paper for insertion and ranking of results.
I love what Norvig said. I can relate to it. As far as data structures are concerned, I think it's worth playing smart with your tests - focus on the "invariants" and ensure their integrity.
A classic example of invariant I can think of is the min-heap - node N is less than or equal to the value of its children - the heap property.
Five years from now, you might forget the operations and the nuanced design principles, but the invariants might stay well in your memory.
> The Perseverance rover has explored and sampled igneous and sedimentary rocks within Jezero Crater to characterize early Martian geological processes and habitability and search for potential biosignatures ..... the organic-carbon-bearing mudstones in the Bright Angel formation contain submillimetre-scale nodules and millimetre-scale reaction fronts enriched in ferrous iron phosphate and sulfide minerals, likely vivianite and greigite, respectively.
> Organic matter was detected in the Bright Angel area mudstone targets Cheyava Falls, Walhalla Glades and Apollo Temple by the SHERLOC instrument ..... A striking feature observed in the Cheyava Falls target (and the corresponding Sapphire Canyon core sample), is distinct spots (informally referred to as ‘leopard spots’ by the Mars 2020 Science Team) that have circular to crenulated dark-toned rims and lighter-toned cores
> PIXL XRF analyses of reaction front rims reveal they are enriched in Fe, P and Zn relative to the mudstone they occur in ..... In the reaction front cores, a phase enriched in S-, Fe-, Ni- and Zn was detected
> Given the potential challenges to the null hypothesis, we consider here an alternative biological pathway for the formation of authigenic nodules and reaction fronts. On Earth, vivianite nodules are known to form in fresh water ..... and marine ..... settings as a by-product of low-temperature microbially mediated Fe-reduction reactions.
> In summary, our analysis leads us to conclude that the Bright Angel formation contains textures, chemical and mineral characteristics, and organic signatures that warrant consideration as ‘potential biosignatures’ that is, “a feature that is consistent with biological processes and that, when encountered, challenges the researcher to attribute it either to inanimate or to biological processes, compelling them to gather more data before reaching a conclusion as to the presence or absence of life .....
> The Planetary Instrument for X-ray Lithochemistry (PIXL) is an X-ray fluorescence (XRF) spectrometer mounted on the arm of the National Aeronautics and Space Administration’s (NASA) Mars 2020 Perseverance rover (Allwood et al., 2020; Allwood et al., 2021). PIXL delivers a sub-millimeter focused, raster scannable X-ray beam, capable of determining the fine-scale distribution of elements in martian rock and regolith targets. PIXL was conceived following the work by Allwood et al. (2009) that demonstrated how micro-XRF elemental mapping could reveal the fine-textured chemistry of layered rock structures of ~3,450-million-year-old Archean stromatolitic fossils. Their work not only pushed back the accepted earliest possible window for the beginning of life on Earth, but also demonstrated that significant science return might be possible through XRF mapping. PIXL was proposed, selected, and developed to carry out petrologic exploration that provide the paleoenvironmental context required in the search for biosignatures on Mars, analogous to Allwood et al.’s earlier work.
I like your analysis, but, personally, I am struggling with "Absence of data/other possibilities is pointing us to conclusion"
It should (IMO) be reported as, we just don't know (yet), there's some really fascinating things that we cannot explain in any other way, yet, but that doesn't actually mean that we know for sure.
What's not to understand - it's precisely the argument theists have put forward for millenia
"We couldn't find anything to show it wasn't a god, so it must be a god"
Calling one group "smart" doesn't change the process or the outcome - the absence of data is not data, it's just that we couldn't yet find the full explanation.
One day we might, it might actually be life, but we don't have that right now, so, actual science demands that we withhold any wild speculation.
No, because theism is missing the first premise. The equivalent would be:
- We have observably seen and reproduced god bringing someone back from the dead
- We can find no other explanation for this thing coming back from the dead
- It was likely god who brought this thing back from the dead, but we want more data
The first premise has never happened, there is not any equivalence...
Actually (It's too late to edit) we do have this curious thing
Aliens - we claim/recognise that statistically the size of the (at least observable, if not entire) universe and number of habitable planets with all the right ingredients for life that there must be life out there... somewhere
But we don't have an ounce of evidence (neither for nor against)
God(s) - we don't have any evidence one way or the other, atheists just say "It's impossible", theists say "It's the only answer", but, as already mentioned, there isn't any actual evidence that can lead us to a conclusion. (This will be misread as an argument for god(s), but it isn't. And even if it were, there's still a massive step between that and the Abrahamic God being the dude)
Which takes me right back to where this started. The supposition that the features of mars are signatures of life, we don't know at this point, all we actually know is... we haven't found anything else that we can say they are.
Also, I wanted to mention something interesting - back when LLM-driven applications were just emerging, someone posted on Hacker News about how they categorized In Our Time episodes using the Dewey Decimal System with LLMs. Cool stuff - https://news.ycombinator.com/item?id=35073603
Shameless plug, but I have a podcast about the history of astronomy and did an hour long episode on the Antikythera Mechanism a little while back if you're interested in more detail: https://songofurania.com/episode/022
I have a question that's bothered me for quite a while now. In 2018, Michael Jordan (UC Berkeley) wrote a rather interesting essay - https://medium.com/@mijordan3/artificial-intelligence-the-re... (Artificial Intelligence — The Revolution Hasn’t Happened Yet)
In it, he stated the following:
> Indeed, the famous “backpropagation” algorithm that was rediscovered by David Rumelhart in the early 1980s, and which is now viewed as being at the core of the so-called “AI revolution,” first arose in the field of control theory in the 1950s and 1960s. One of its early applications was to optimize the thrusts of the Apollo spaceships as they headed towards the moon.
I was wondering whether anyone could point me to the paper or piece of work he was referring to. There are many citations in Schmidhuber’s piece, and in my previous attempts I've gotten lost in papers.
Thanks! This might be it. I looked up Henry J. Kelley on Wikipedia, and in the notes I found a citation to this paper from Stuart Dreyfus (Berkeley): "Artificial Neural Networks, Back Propagation and the Kelley-Bryson Gradient Procedure" (https://gwern.net/doc/ai/nn/1990-dreyfus.pdf).
I am still going through it, but the latter is quite interesting!
> "Since his first work on the subject, the author has found that A. Bryson and Y.-C. Ho [Bryson and Ho, 1969] described the backpropagation algorithm using Lagrange formalism. Although their description was, of course, within the framework of optimal control rather than machine learning, the resulting procedure is identical to backpropagation."
Apologies - I should have been clear. I was not referring to Rumelhart et al., but to pieces of work that point to "optimizing the thrusts of the Apollo spaceships" using backprop.
One thing AI has been great for, recently, has been search for obscure or indirect references like this, that might be one step removed from any specific thing you're searching for, or if you have a tip-of-the-tongue search where you might have forgotten a phrase, or know you're using the wrong wording.
It's cool that you can trace the work of these rocket scientists all the way to the state of the art AI.
I don't know if there is a particular paper exactly, but Ben Recht has a discussion of the relationship between techniques in optimal control that became prominent in the 60's, and backpropagation:
Rumelhart et al wrote "Parallel Distributed Processing"; there's a chapter where he proves that the backprop algorithm maximizes "harmony", which is simply a different formulation of error minimization.
I remember reading this book enthusiastically back in the mid 90s. I don't recall struggling with the proof, it was fairly straightforward. (I was in senior high school year at the time.)
To be fair, any multivariable regulator or filter (estimator) that has a quadratic component (LQR/LQE) will naturally yield a solution similar to backpropagation when an iterative algorithm is used to optimize its cost or error function through a differentiable tangent space.
I believe the reason it works in nonlinear cases is that the derivative is “naturally linear” (to calculate the derivative, you are considering ever smaller regions where the cost function is approximately linear - exactly “how nonlinear” the cost function is elsewhere doesn’t play a role).
What is "this" exactly? Is it a well-known author or website? Or otherwise a reference that one should be familiar with? It looks like a random blog to me... with an opinion declared as fact that's quite easy to refute.
Because it is terribly low-effort. People are here for interesting and insightful discussions with other humans. If they were interested in unverified LLM output… they would ask an LLM?
Who cares if it is low effort? I got lots of upvotes for my link to Claude about this, and pncnmnp seems happy. The downvoted comment from ChatGPT was maybe a bit spammy?
It's a weird thing to wonder after so many people expressed their dislike of the upthread low-effort comment with a down vote (and then another voiced a more explicit opinion). The point is that a reader may want to know that the text they're reading is something a human took the time to write themselves. That fact is what makes it valuable.
> pncnmnp seems happy
They just haven't commented. There is no reason to attribute this specific motive to that fact.
I don't think it's rude, it saves me from having to come up with my own prompt and wade through the back and forth to get useful insight from the LLMs, also saves me from spending my tokens.
Also, I quite love it when people clearly demarcate which part of their content came from an LLM, and specifies which model.
The little citation carries a huge amount of useful information.
The folks who don't like AI should like it too, as they can easily filter the content.
> ... first arose in the field of control theory in the 1950s and 1960s. One of its early applications was to optimize the thrusts of the Apollo spaceships as they headed towards the moon.
I think "its" refers to control theory, not backpropagation.
One of my favorite examples of this is the Beatles:
> On 11 February 1963, the Beatles recorded ten songs during a single studio session for their debut LP, Please Please Me.
> Martin asked the band if they had any songs that they could record quickly. According to Martin, "It was a straightforward performance of their stage repertoire – a broadcast, more or less." Initially, a morning and afternoon session only were booked; the evening session was added later. Mark Lewisohn later wrote: "There can scarcely have been 585 more productive minutes in the history of recorded music".
Interestingly,
> .... the Beatles arrived with John Lennon suffering from a bad cold, which he attempted to treat with a steady supply of throat lozenges
Regarding the intersection takeoff, Flightradar24 just tweeted this:
> We are continuing to process data from receiver sources individually. Additional processing confirms #AI171 departed using the full length of Runway 23 at Ahmedabad. RWY 23 is 11,499 feet long. The aircraft backtracked to the end of the runway before beginning its take off roll.
I was thinking the same thing! I've been working with some TTS applications, such as real-time commentary for Pong and personalized radio stations. I might give this a try, it sounds fun.
I have described it in the "Timer Modules" section:
> A natural iteration of this approach is to store timers in an ordered list (also known as timer queues). In this scheme, instead of storing the time interval, an absolute timestamp is stored. The timer identifier and its corresponding timestamp that expires the earliest is stored at the head of the queue. Similarly, the second earliest timer is stored after the earliest, and so on, in ascending order. After every unit, only the head of the queue is compared with the current timestamp. If the timer has expired, we dequeue the list and compare the next element. We repeat this until all the expired timers have been dequeued, and we run their expiry processing routines.
And then, I go on to talk about its runtime.
Truth be told, this is a chapter for my book on data structures and algorithms that I think are interesting and obscure enough that not many people talk about them. Its goal is not widespread practicality, but rather a fun deep dive into some topics.