Re-fielding Replicability of Silicon Sample-based Marketing Research

Large language models are increasingly used to generate silicon samples, synthetic respondents that stand in for human participants in behavioral research. As silicon samples enter the methodological mainstream, the question of their re-fielding replicability becomes critical: when researchers regenerate a silicon sample under the same procedure at a later point in time, how stable are the resulting data? Existing validation studies have compared LLM-generated responses to human responses at a single point in time, but the temporal consistency of silicon samples themselves (whether independent reapplications of the same generation procedure yield comparable samples) has not been examined systematically, especially under sparse-conditioning, re-fielded sample-generation settings. We address this gap by generating silicon samples across three weekly waves, permitting a re-fielding consistency assessment. We benchmark this assessment with a human reference sample recruited via Prolific. Moreover, we systematically compare three structurally distinct generation paradigms that try to increase response variability: single-prompt batch generation, iterative generation with cumulative memory, and two-stage seed-and-expand generation. We find that re-fielding consistency is reasonably high in aggregate and comparable to the human benchmark for the strongest paradigm (two-stage seed-and-expand) but markedly lower for single-prompt batch generation. In contrast, alignment with the human reference is generally low, so that silicon samples reproduce their own outputs more faithfully than they reproduce human data. This means that silicon samples can be internally reproducible without being externally valid.