Are multiple LLM queries faster than vector search? Even with the example "dog OR canine" that leads to two LLM inference calls vs one. LLM inference is also more expensive than vector search.

In general RAG != Vector Search though. If a SQL query, grep, full text search or other does the job then by all means. But for relevance-based search, vector search shines.