In assessment, activity types have often been constrained by the technology available to administer and score them. Researchers have previously explored the possibilities of conversation-based assessment, which would engage learners in dialogue to generate evidence about what they know and can do. This approach aims to emulate the conversations teachers have with students as they probe their thinking about problems. Unfortunately, the technology available at the time made design and control of conversations difficult. However, recent advances in generative artificial intelligence (AI) now remove many of these earlier barriers. We now have a new opportunity to explore this space with the availability of generative AI.