Bongo AI Coach FAQ

Follow

The Bongo AI coach is designed to provide tonal analysis and coaching tips for learner videos.  These will help drive practice opportunities for learners in a safe-space.  Once videos have been submitted for review, instructors can use the AI tips and analysis to cover the basics while they can focus on providing feedback based on their subject-matter expertise.

Administrators can also generate Learning Objectives which will be used to provide targeted coaching tips and to optionally be used to automatically score a learner’s video.

What is the underlying technology for Bongo AI Coach?
The Bongo AI Coach utilizes a large-language model and generative AI to produce coaching tips and tonal analysis. In the first release, the Bongo product will be interfacing with the Open AI GPT-3.5 model though new versions and other models may be considered in the future as the technology evolves.
How is the Bongo AI Coach different from ChatGPT?

While ChatGPT utilizes the same underlying AI model that the Bongo AI Coach will be using, there are key differences that users can expect. ChatGPT is an open dialog and can take a significant amount of trial-and-error to get the desired results. It also has limitations on how much data a user can send in.

The Bongo product will be building prompts that are tailored to our use case of providing learners with actionable feedback directly from within the Bongo product. Users will never need to enter prompts to get the results they need. Bongo will also handle complex prompt engineering that will allow for feedback on videos that would otherwise exceed the limits one would encounter when interfacing directly with ChatGPT.

What are the different elements of Tone Analysis?

The Bongo AI Coach will automatically assess the tone of a learner’s video submission across the following four dimensions:

  • Confident vs. Nervous
  • Persuasive vs. Informative
  • Humorous vs. Serious
  • Casual vs. Formal

The coach will provide insight as to where a video sits on the ranges above and provide examples that reinforce that analysis.

What can I expect from Coaching Tips and Feedback?
The Bongo AI Coach will review a learner’s video and share tips and feedback around, opportunities for improvement, areas of excellence, and unique facets to what they have presented. The Bongo AI Coach will provide specific examples from a learner’s transcript to support each of those areas.
What information is Bongo sending to OpenAI?
In order to retrieve coaching tips and tonal analysis, Bongo will be sending the video transcript and Auto Analysis metrics. The actual video and any information about the user will not be sent.
How is my personally identifiable information (PII) being protected?
No PII will be directly sent to OpenAI's GPT model, only data related to the video transcript generated by AWS Transcribe. The transcript data will be encrypted both in transit and at rest. Bongo will send the data to a version of the model that is appropriately located relative to a user’s region. Finally, by using OpenAI models through an API, the transcript will not be used for training the models which will avoid exposure to proprietary or private information.
How does the Bongo AI Coach handle bias?
The Bongo team is aware that AI models can be subject to bias based on the data set on which they have been trained. While there is no magic bullet, we are building in some safety measures. First, closely controlling how the OpenAI model is prompted will ensure more consistent results as compared to an open-ended experience. In the first release and until further testing confirms reliability, we are relying on a separate service to transcribe the speech in the video and will not be sending the actual audio track to provide an objective assessment.
Can I turn off the Bongo AI Coach?
If your institution or organization does not want the AI Coach activated for their learner videos, please contact our support team to discuss options for turning it off.
Can I customize the tonal ranges evaluated by the Bongo AI Coach?
For the initial release, all users will have a standard set of 4 tonal ranges that the AI Coach will evaluate. The AI Coach is an area of strategic investment for the Bongo team and the ability to customize tonal ranges is an item for future consideration.
Can I customize how the Bongo AI Coach will review videos for my assignment?
Yes. By providing course materials at the time of assignment configuration to develop Learning Objectives, the AI Coach will provide feedback specifically around how well those Learning Objectives have been met in addition to the standard coaching feedback.
Does the Bongo AI Coach results impact the Auto Analysis score?
The coaching tips and tone analysis produced by the Bongo AI Coach are designed to be useful insights on areas of improvement, but they will not impact the Auto Analysis score initially.
If I record multiple videos for an assignment, will the Bongo AI Coach evaluate them against one another?
No. The Bongo AI Coach will provide feedback on each video in isolation.
Are there any situations where I will not get Tips, Feedback, and Tone Analysis from the Bongo AI Coach?
If there are fewer than 30 words in your video, Tips, Feedback, and Tone Analysis will not be available as the AI model does not have enough information to produce quality results in those circumstances.
Can I create my own Learning Objectives or edit those Bongo creates for me?
Currently you cannot add or edit Learning Objectives directly. The algorithm used to generate learning objectives is designed to create ones that can be effectively used to build targeted coaching tips and produce a Smart Score result. However, you can run through the Learning Objective generation process multiple times and adjust input text as necessary to produce the objectives that are best suited to your assignment.
Does the Auto Analysis score impact the Smart Score results?
No. Those scores are entirely independent. If you choose Smart Scoring as your Evaluation Type, we will automatically hide the Auto Analysis scores from learners to avoid any confusion.
How reliable are the Smart Score results?
The Bongo team has built tools to run the Smart Score algorithm at high-scale. By repeatedly running the same videos, the scoring results were confirmed to have reliability within targeted guidelines similar to that of human evaluation of subjective matter.
How accurate are the Smart Score results?
Smart Scoring has undergone a battery of tests to compare its results with those of videos assessed by human instructors. At this time, the Smart Score results typically fall within 5% of the human evaluations.
Can I make adjustments to the Smart Score results?
Yes. Instructors have the ability to make adjustments to the score provided for each Learning Objective. These adjustments will be tracked to inform future improvements to Bongo’s AI capabilities. A flag is available at the time of configuration to manage learner access to Smart Score results. Results can either be immediately published and available to learners or they can be hidden from learners until an instructor reviews and makes adjustments if necessary.
Can I turn off Smart Scoring?
If your institution or organization does not want Smart Scoring available as an evaluation type, please contact our support team to discuss options for turning it off.
How many Learning Objectives and criteria can I have?
There are no defined limits in the system on the number of learning objectives or criteria. However, testing seems to indicate that the time required to complete scoring when there are more than 8 criteria under a specific learning objective can increase quite a bit due to multiple cycles of querying the language model.
8 out of 8 found this helpful

Comments

0 comments

Article is closed for comments.