Some Zen teachers think that it is impossible to meditate while walking as it keeps the mind moving rather than still. These are the folks that go against any kind of seasoning in food for the same reason. I always thought that was a very restrictive way to box in and needlessly constrain what meditation can be. If it works for you, great but don't sell it as the only path. That is the thing with a lot of folks, to try and overly define 'the only way', the smarter ones know there is a thousand paths to the top of the mountain.
Thích Nhất Hạnh used to swear by walking meditation, others would scoff at that. Each to their own.
'There is a thousand paths to the top of the mountain, the view is the same for all at the peak'
Rumi (the Sufi mystic) apparently walked and turned in circles in order to contemplate. The tradition merges music and movement with philosophy and religious mysticism.
Walking, dancing or manual labor (for example gardening or cleaning) can all be done in a meditative way.
But these are likely different types of meditation that have different effects. Even just a calm, sitting meditations might be vastly different from another, depending on the meditation object.
Of course there are people who lean into specific types over the others as you describe, but I think many of these activities share a common core and experience.
I buy it. I'm not really into meditation, but am deep thinking/reflection.
I found I got by far the most intense deep thinking sessions while mowing the lawn with a push mower. It was a large-ish yard, took around an hour. It's boring, monotonous, requires no thought. Keeps your hands occupied so you won't be tempted to 'check something real quick'. And lastly, loud enough to block any other sounds that could make your mind drift(sirens, birds, dogs barking, etc).
Yes, everything can be mediatative but it's more the matter of how you do it. You can do gardening but still renumerate about past and future. It's all about focusing on what you're doing and nothing else, centralising all 6 senses to one point of fucus.
There’s a lot of research on restorative environments (usually nature/outside)being good for focus. I definitely try to spend as much time outside as I can, but for some reason the wall works better for that 5-10 minutes. Being outside is much more enjoyable though haha
Interesting. I hadn't heard of that directly, but I've never found it to be true. I've found momentum and continuation to be more useful than rest or relaxation when it comes to tackling big things.
"a walk with just your thoughts" can lead you to many places include good and bad places. A mediative walk is to focus on nothing else but your body including your breath, your arms, your steps, the sensation of skins, the smell you breath in and out, the sound you hear. You don't need to focus on all of these at once but you can just pick one and focus on it for a period ideally as long as possible, but you can switch between them at the start. All of these is very difficult but not impossible if you do in a crowded city when everything changes very fast and you're exposed to many things that you tend to like or dislike. Of course, if you're a zen master, you can meditate everywhere but personally I feel I'm very far from it. The fact that I struggle to focus even in quiet places tells a lot about how much I need to practice.
The underlying C library interacts directly with the postgres query parser (therefore, Postgres source). So unless you rewrite postgres in Rust, you wouldn't be able to do that.
Well then why didn’t they just get the LLM to rewrite all of Postgres too /s
I agree that LLMs will make clients/interfaces in every language combination much more common, but I wonder the impact it’ll have on these big software projects if more people stop learning C.
grug very elated find big brain developer Bob Nystrom redeem the big brain tribe and write excellent book on recursive descent: Crafting Interpreters
book available online free, but grug highly recommend all interested grugs purchase book on general principle, provide much big brain advice and grug love book very much except visitor pattern (trap!)
Grug says bad.
In all seriousness, the rough argument is that it's a "big brain" way of thinking. It sounds great on paper, but is often times not the easiest machinery to have to manage when there are simpler options (e.g. just add a method).
It's not bad if you need something quick. I haven't had a large need of ANN in duckdb since it's doing more analytical/exploratory needs, but it's definitely there if you need it.
Just curious what the state of the art around filtered vector search results is? I took a quick look at the SPFresh paper and didn't see it specifically address filtering.
In any API service, it's better to handle via dependency injection IMO.
Instantiate all of your metadata once, and then send that logger down, so that anybody who uses that logger is guaranteed to have the right metadata... the time to add logging is not when you are debugging.
I don't disagree that rock solid is a good choice, but there is a ton of innovation necessary for data stores.
Especially in the context of embedding search, which this article is also trying to do. We need database that can efficiently store/query high-dimensional embeddings, and handle the nuance of real-world applications as well such as filtered-ANN. There is a ton of innovation in this space and it's crucial to powering the next generation architectures of just about every company out there. At this point, data-stores are becoming a bottleneck for serving embedding search and I cannot understate that advancements in this are extremely important for enabling these solutions. This is why there is an explosion of vector-databases right now.
This article is a great example of where the actual data-providers are not providing the solutions companies need right now, and there is so much room for improvement in this space.
I do not think data stores are a bottleneck for serving embedding search. I think the raft of new-fangled vector db services (or pgvector or whatever) can be a bottleneck because they are mostly optimized around the long tail of pretty small data. Real internet-scale search systems like ES or Vespa won’t struggle with serving embedding search assuming you have the necessary scale and time/money to invest in them.
* Filterable ANN certainly decomposes into pre- and post-filtering, and there is definitely a lot of interesting innovation occurring around filterable ANN. But large-scale search systems currently do a pretty good job with pre-filtering, falling back to brute force search in the case of restrictive filters.
* You'd have to be a bit more exact re: dynamic updates/versioning for me to understand the challenges you're facing.
* Building graph indices can be slow, but in my experience (billions of embeddings) it is possible to build HNSW indices in tens of minutes.
* How is this any different to combining traditional keyword search with, say, recency boosting?
Might be missing my argument here - I stated that there are workable solutions to this like you have pointed out.
But ANN search is still a sledgehammer and building out hybrid solutions that help bridge the gap between this and traditional data stores still have room for innovation.
Fair enough - agreed there's lots of interesting innovations here - but my point is that semantic search and its associated issues don't really differ that much from other types of search problems at scale, and I therefore don't think that the current crop of vector database products add a lot of value from a technical perspective (perhaps they do from an ease-of-use perspective; or they work great at small scale, etc. etc.)
Oh, then you must have the secret sauce that allows scaling ES vector search beyond 10,000 results without requiring infinite RAM. I know their forums would welcome it, because that question comes up a lot
Or I guess that's why you included the qualifier about money to invest
Would you mind putting aside the snark? I have a couple questions. How large is the corpus? I am also curious about the use-case for top-k ANN, k > 10000?
Not the person you have asked but at work (we are a CRM platform) we allow our clients to arbitrarily query their userbase to find matching users for marketing campaigns (email, sms, whatsapp). These campaigns can some times target a few hundred thousand people. We are on a really ancient version of ES, but it sucks at this job in terms of throughput. Some experimenting with bigquery indicates it is so much better at mass exporting.
Fair; my question was mostly in the context of ANN, since that was the discussion point - I have to assume ES (as a search engine) would not necessarily be the right tool for data warehousing types of workloads.
Yes, that means no phone, no headphones, just you and your brain enjoying a walk. Let your mind wonder and be free.