The general availability of powerful Large Language Models had a powerful impact on higher education, yet general models may not always be useful for the associated specialized tasks. When using these models, oftentimes the need for particular domain knowledge becomes quickly apparent, and the desire for customized bots arises. Customization holds the promise of leading to more accurate and contextually relevant responses, enhancing the educational experience. The purpose of this short technical experience report is to describe what “customizing” Large Language Models means in practical terms for higher education institutions. This report thus relates insights and experiences from one particular technical university in Switzerland, ETH Zurich.