What is notably missing from the AlphaCode system is any architectural design in the ML model that relates to the task of interest: producing code. Computer code is a highly structured medium; programs must adhere to a defined syntax and must produce well-defined pre- and postconditions within the different portions of the solution. Program synthesis has a long history (6) and numerous techniques have been developed for generating programs that obey these types of constraints. It seems only natural that given a medium as structured as computer code, this structure would be used by ML models aiming to write code. But AlphaCode does none of this. It generates code the way LLMs generate any text— one token at a time—and only checks for program correctness after the entire program has been written. It may seem surprising that this procedure has any chance of creating correct code. But the reality is that given the proper data and model complexity, coherent structure can emerge.