Cogniti is best implemented at a school or institution level. This does not mean that you need to have everyone using it at once, but rather that AI platforms are best considered as a strategic investment by educational institutions.
Effective implementations usually start with a small pilot of 5-50 educators with their classes/cohorts, thinking about how to use Cogniti for particular pedagogical purposes.
Access and integration #
The first step is to get access to Cogniti. There are a number of ways for your institution to do this:
- DIY/self host – sign a licence with the University of Sydney and run Cogniti in your own Azure environment
- Azure Marketplace (coming soon) – easily install a stable version of Cogniti from the Marketplace and receive automatic updates
- Software-as-a-service – the University of Sydney hosts Cogniti for your institution
It’s important to involve your IT team to provision the platform and enable login permissions and access for educators and students. You will also need to ensure you follow necessary local governance processes.
You should also consider LMS integration – this allows educators to embed Cogniti agents and other assets directly into the LMS (like Canvas and Moodle, but Cogniti also currently supports Brightspace and Schoolbox, and building support for other LMSs is quick) so that it’s straightforward for students to access without separate login, through LTI 1.3. For Canvas institutions, you can also connect Cogniti to Canvas via its API to pull assets from Canvas to inform Cogniti’s responses.
Training for educators #
We usually recommend that educators new to Cogniti attend a 1 hour training session. We find this important because it helps people to understand the philosophy behind Cogniti (including that it is there to augment, not replace, teachers) and get an introduction to designing custom AIs. The University of Sydney runs regular training sessions for internal and external educators, and you are welcome to adapt our material if you’d like to run your own. We may also be able to run a bespoke training session for your institution if there is sufficient interest – please get in touch.
The typical flow of a training session that we have found works well includes:
- Brainstorming some ideas around ‘AI doubles’
- Showing two agents in action
- Explaining some of the background about how agents work, what a system message looks like, and caveats around resources
- Demonstrating the live build of a simple agent using an idea from a participant
- Time to build an agent (about 5-10 minutes)
- Q&A
It may also be a good idea for learning designers and other staff who will be providing local support to be trained up on common challenges that arise. We can help run these sessions for you.
Providing a link to this site that you are on (the Cogniti website and its documentation; https://cogniti.ai/docs/) is a good idea. Also, colleagues from the University of Sydney have written up a straightforward and comprehensive guide to getting started with Cogniti:
Introducing AI to students #
After building an agent, it’s very important to introduce it to students properly. This includes explaining why you built the agent for them, how they can access it, and how to best use it to support their learning. We have seen poor engagement in situations where students are not properly introduced to the AI agents that their educators have built.
Here are some ideas for sharing your agent with students.
Looking after educators and their agents #
Once agents are released, it’s important to recognise that it should not be set-and-forget. Just as you would have regular catch-ups with teaching assistants or tutors working with you, you should ‘check in’ with your agent regularly. Some ideas:
- Check the conversation history to understand how your students are interacting with your agent
- Use agent insights to show a quick summary of these interactions
- Talk with your students about how they are finding the agent and whether it is contributing to their learning, and why
It is important to iterate on the design of your agent. For example, amend its system message based on desirable and less desirable behaviours you may be observing. You may find that it’s better to remove some resources and instead focus on putting more specific context information into the system message.
For educators who have developed some experience with Cogniti, it is often helpful to run masterclasses or communities of practice to troubleshoot, share, and learn. When we run masterclasses, we often cover the following topics. Again, we may be able to run these sessions for you if there is sufficient interest.
- Advanced prompting techniques
- ‘Agent clinic’ where educators present their agents and get some help in improving its behaviour
- Q&A
Local instructional/learning design teams play an important role in supporting and checking in with educators and their agents, to help ensure that the AI benefits student learning and experience. These teams can help align the technology with intended pedagogy, host communities of practice, run consultations and training, and more.
Updates and ongoing support #
New updates to the Cogniti codebase occur regularly, and these include feature updates as well as bug fixes. If you are DIY or self-hosting Cogniti, you will need to arrange with your IT team to bring in these updates. Marketplace and SaaS users will receive these updates automatically.
Your local learning design, instructional technology, and IT teams will likely be best placed to support Cogniti locally. They will have the local knowledge, examples, relationships, and educational technology know-how to provide this support. These teams are welcome to reach out to us if they have any questions.
Evaluation #
It will likely be important to evaluate the impact of Cogniti on educators and students. Any evaluation will clearly be dependent on how Cogniti is being used. Evaluation might involve considering:
- Whether students are more engaged
- Impact on learning outcomes and performance
- Different pedagogical approaches enabled by Cogniti
- How educator time is spent differently
- How AI use impacts on student metacognitive skills
- Satisfaction of educators and students
- Engagement metrics reported by Cogniti itself
- Reviewing conversation histories to discover how and why students use your agents
There are many educational institutions using Cogniti who are exploring evaluation. Please contact us if you would like to gain access to the Cogniti Across Institutions Microsoft Teams site. We have almost 200 people from over 60 institutions on it sharing best practices around use and evaluation.
