An experiment report on generating short-form video ideas by independent sampling across five frontier language models, evaluated in embedding space against a labelled corpus of previously published work.
Note on this edition. Published on the AI Creator Adda site, with two changes from the original. Costs are omitted: the site prints no spend figures, so the comparison between models is stated in themes rather than dollars. And section 4.2 is corrected — an earlier draft reported a twelve-draw convergence across every vendor on the MIS Analyst role, which this run's own analysis.json does not support.
A short-form creator publishing roughly one reel a day needs a continuous supply of ideas, and the supply is the bottleneck rather than the production. We tested whether sampling many independent draws from several frontier models, rather than iterating within one conversation, produces a usable and genuinely varied idea pool. One idea was drawn per API call, 30 calls per model across five models, each call entered through one of 20 rotating angles to prevent mode collapse. The 150 ideas were embedded with a 3072-dimensional model and clustered.
Every call returned a usable idea. The pool resolves into themes. Ideas sit further from the creator's existing work (mean cosine similarity to the nearest of 34 posted reels) than his own reels sit from each other (), so the generator is opening new ground rather than rediscovering the back catalogue. Separating the two effects the design confounds: at a fixed budget of 30 draws, spreading them across five models beats one model by themes ( against ), a real but modest gain, while raising the budget from 30 draws to 150 moves coverage from themes to . Scale mattered more than model mixture. We measured diversity and novelty, not quality; no human has yet rated whether these ideas perform.
Table 1. Summary of principal results. Each quantity is defined and derived in the section named.
| Quantity | Value | Where |
|---|
The creator behind @your_data_scientist posts a teaching reel most days to an Indian audience learning AI and data science. A companion analysis of 35 reels posted between 28 June and 1 August 2026 found a median of 1,039 views against two breakouts at 34,405 and 58,264. The breakouts shared a shape: they reframed a job title the viewer already holds, and attached a one-word comment keyword to a real deliverable. The floor shared a different shape: eight reels between 458 and 672 views, five of which took exactly zero comments, each explaining a term or reporting a model launch without asking the viewer to decide anything.
That gives an unusually well-specified ideation problem. We know what the target looks like, we know the dominant failure mode, and we have a labelled corpus of prior attempts to measure novelty against. The open question is procedural: is it better to think hard once, or to sample widely and select?
This report contributes four things. A prompt built from measured performance rather than intuition, and a lens-rotation scheme that prevents the sampling collapse such a setup normally suffers. A measurement of how many distinct themes 150 draws actually yield. A novelty test against the creator's own back catalogue, with the baseline needed to make its number mean anything. And a separation of the two variables the design confounds, sampling scale and model mixture, which turns out to reverse the experiment's founding assumption.
One prompt, roughly 5,700 characters, holding the creator's mission, the measured performance of all 35 prior reels with real view counts, and the rules distilled from them. Two choices in it are load-bearing. It leads with the losers' shared mechanic rather than the winners', because "five of the eight worst reels took zero comments" is a more correctable instruction than "career topics win". And it explicitly retires the winning sentence: the 58k reel's construction is named as used up, so the models reuse the mechanism underneath it instead of producing 150 variants of one headline.
One idea per call, not a list of ten. A model asked for ten ideas optimises the set, padding it after the first strong entry; one idea per call makes each draw independent, which is what makes the 150 rows comparable and the saturation analysis meaningful.
Thirty identical prompts at temperature 1.0 do not explore a space. Models have strong modes and re-sample around them, so the draws collapse onto a handful of ideas in different words. Each call therefore enters through one of 20 rotating lenses, a required angle of attack (the failure mode, the verification problem, the career door outside software, the thing a senior will not tell a junior). Call n uses lens n mod 20, so every model covers the full set at least once.
Five models through OpenRouter, 30 draws each, temperature 1.0, reasoning enabled, structured JSON output with a parse retry. Two configuration details cost real debugging and are worth recording. Setting provider.require_parameters, the pattern used elsewhere in this codebase to pin routing to schema-honouring providers, removes every viable endpoint for some vendors: Claude Opus 5 returned a 400 for unsupported structured outputs and GPT-5.6 Sol a 404 for no endpoints found. And reasoning tokens bill against max_tokens, so an initial ceiling of 1,200 was consumed entirely by DeepSeek's reasoning, returning empty content that looks exactly like a parse bug. The ceiling was raised to 6,000 after measuring Grok at 3,958 tokens on this prompt.
All 150 ideas and all 34 recoverable prior reels were embedded with OpenAI text-embedding-3-large at 3072 dimensions, then row-normalised so a dot product is cosine similarity. Ideas were clustered with agglomerative clustering on cosine distance, average linkage, the cluster count chosen by silhouette over . Cluster labels come from one model call per cluster; exemplars are medoids, meaning a real idea nearest the cluster centre rather than a synthesised description. Every random component takes a fixed seed.
150 of 150 calls returned a parseable, non-empty idea.
Silhouette selected clusters. The peak is shallow (, with neighbouring values within 0.004), which is normal for short-text embeddings and means the exact count should be read as "roughly this many themes", not as a discovered constant. Of these, contain ideas from more than one model and are singletons: one idea no other draw came near.
For each generated idea we took the cosine similarity to its nearest prior reel. The mean is , the maximum , the minimum .
That number is meaningless on its own. A similarity of could be far or near depending on how this embedding model scores any two texts from one narrow niche, so the reference that matters is how similar the creator's own reels are to each other. They are much more alike than any generated idea is to them.
Table 2. Nearest-neighbour cosine similarity within and between the two populations. The first row is the baseline that makes the third interpretable.
| Nearest-neighbour cosine similarity | Mean |
|---|
A posted reel's nearest neighbour among the other posted reels scores . A generated idea's nearest posted reel scores only . The generator is therefore working outside the creator's existing territory, by a margin larger than the territory's own internal spread. That is the result the raw number was trying to claim, and it survives the control.
The same table carries a caution the headline hides. Generated ideas are more similar to each other () than posted reels are to each other (). The pool is novel with respect to prior work while being internally tighter than the creator's real range. That is a predictable consequence of the prompt: 20 lenses all pointed at agentic systems, so the output is concentrated on-mission rather than spanning everything he actually posts about. Read as "a deep seam next to the existing catalogue", not "a broader catalogue".
This is the result the experiment was run to get. Resampling the draw order 400 times gives the expected number of distinct themes found after n draws.
The marginal return falls sharply and legibly. The first 30 draws yield themes, about per draw. Draws 31 to 75 add more, about per draw. Draws 76 to 150 add , about per draw. A run half this size would have found roughly of the themes for half the money.
Read the two curves as answering different questions. The gap at 30 draws ( themes) is what mixing vendors buys at a fixed budget. The rise along the pooled curve is what buying more draws buys. The second is much larger than the first, which is the opposite of what this experiment was set up expecting; section 4.1 works through it.
Within-model similarity () exceeds across-model similarity (). The margin is small but consistent, and it means each model has a recognisable signature: its own draws resemble each other more than they resemble another vendor's. That is the mechanism behind the mixture gain in 4.1.
Note that the per-model totals below are each from 30 draws and are not comparable to the found by the pool, which had 150. They compare models to each other, not models to the pool.
Table 3. Per-model contribution. Each model drew 30 ideas, so the theme counts are comparable to each other but not to the 37 found by the pooled 150 draws.
| Model | Themes reached | Internal similarity | Similarity to prior work |
|---|
DeepSeek V4 Flash reached themes. Claude Opus 5 reached . On this task, at this sample size, the least expensive model in the set returned more distinct themes than the most expensive one. If this experiment is repeated, the efficient configuration is to scale draws on a cheap model and keep one expensive model in the mix for contrast, rather than to spread the budget evenly.
The clusters, largest first. Sizes are uneven by a factor of ten: the head of the distribution is where several models converged on the same territory, and the tail is where a single draw found something on its own. Both ends are useful for different reasons, the head as consensus and the tail as reach.
Table 4. The 37 themes recovered by clustering, ordered by size. "Models" is the number of distinct models that reached the theme; the exemplar is the medoid, the real idea closest to the cluster centre.
| # | Theme | Ideas | Models | Exemplar (medoid) |
|---|
The design confounds two variables, and the tempting summary ("one model reached themes, five models reached ") silently compares 30 draws against 150. Figure 3 separates them, and the honest accounting is less dramatic than that sentence.
Mixture, at fixed budget: 30 draws spread across five models find themes; 30 draws on one model find on average. Pooling buys themes, about more coverage for the same money. Real, worth having, not transformative.
Scale, at fixed mixture: going from 30 draws to 150 takes coverage from to themes. That is the larger effect by a wide margin.
So the headline is that scale mattered more than model choice, with mixture a secondary bonus. This inverts the intuition the experiment was designed around. If a budget is fixed and something must give, take more draws before taking more vendors. The practical caveat is that mixture is nearly free to add (it is a list of model ids) while scale is linear in cost, so there is no reason to give up the .
Why mixture helps at all is visible in the similarity numbers: within-model similarity () exceeds across-model similarity (). Each model has a signature and re-samples near it, so a second vendor lands in slightly different territory than a 31st draw from the first would.
The lens rotation appears to be doing the work that a naive setup would have missed. A companion word-overlap analysis of the same 150 rows found zero cases of a model repeating itself, and all 20 lenses were represented in the output. Mode collapse was the expected failure of this design, and it did not occur. The cost of that insurance was writing 20 lines of prompt.
Where the models converged is informative rather than embarrassing. Two themes were reached independently by all five vendors: AI agent loop analogies (8 ideas) and AI Data Literacy (7). No model was pointed at either. Five vendors landing separately on the same ground suggests these are optima of the evidence itself rather than quirks of one model's training. Against that, 14 of the 37 themes came from a single model, which is the clearest argument in the run for not standardising on one vendor.
The question was whether to think hard once or sample widely and select. On this task the answer is sample widely, with two corrections to the naive version of that advice.
The width that matters is draws, not vendors. The experiment was built on the assumption that mixing five frontier models was the lever. It is not. At a fixed budget of 30 draws, mixing buys extra themes over a single model ( against ), roughly . Multiplying the draws fivefold buys . Both are real; they are not the same size. Mixture survives as a recommendation only because it is nearly free to add, being a list of model ids, while draws cost money linearly.
The sampling needs structure or it does not scale. Thirty identical prompts at temperature 1 re-sample one region. The 20 rotating lenses cost twenty lines of prompt and produced zero within-model repetition across 150 draws, with every lens represented. Without that, this experiment would have measured the cost of rediscovering the same four ideas thirty times. Any attempt to replicate the yield reported here without a diversification scheme should expect a fraction of it.
What the pool is. distinct themes, sitting genuinely outside the creator's existing work: an idea is further from its nearest posted reel () than his reels are from each other (). It is a deep seam rather than a wider catalogue, because all 20 lenses point at agentic systems. For zero failed calls out of 150, that is a favourable trade against the alternative of a creator staring at a blank page.
What this does not establish. Every measurement here is of the pool's shape, not its worth. Distinctness and novelty are necessary conditions for a useful idea bank and nowhere near sufficient ones, and the selection half of "sample widely and select" was never tested: nothing in this work ranks the themes by expected performance. A generator that reliably produces novel, well-separated, useless ideas would score identically on every statistic in this report. Until reels get shot, the honest claim is that inference-time scaling solved the supply problem and left the selection problem open.
No formal literature review was carried out; this is an internal experiment report, not a survey, and it makes no claim to novelty against published work on sampling or ensembling. The methods below are standard and are cited so the choices can be checked rather than taken on trust.
text-embedding-3-large, 3072 dimensions. Used for all embeddings in Section 3.Model behaviour is provider-dependent and is not guaranteed stable over time, which bounds reproducibility of items [4] and [5] to the cached artifacts rather than to fresh calls.
One set of numbers in this document comes from outside the analysis pipeline and is stated as measured rather than derived: the prior-reel view counts, pulled from Instagram through the Apify Reel Scraper on 2 August 2026. View counts are a snapshot and keep climbing after collection.