OpenAI’s Sora Makes Significant Strides in Generative AI, but Regulators Need to Catch up Without Overly Relying on Self-Regulating Measures Like the C2PA

Subscribe To Download This Insight

By Benjamin Chan | 4Q 2024 | IN-7652

OpenAI’s Sora, with its text-to-video Large Language Model (LLM), opens a realm of possibilities and unlocks more use cases for Generative Artificial Intelligence (Gen AI). However, while OpenAI attempts to self-regulate and ethically manage its product, regulators should not solely rely on market leaders to regulate the Artificial Intelligence (AI) market, as they cannot exert strong influence over bad actors in the space with limited regulatory powers.

Registered users can unlock up to five pieces of premium content each month.

Log in or register to unlock this Insight.

 

OpenAI's Sora Launches with Safeguards to Manage Potentially Harmful Effects of Gen AI

NEWS


In December 2024, OpenAI launched its latest breakthrough in Generative Artificial Intelligence (Gen AI), Sora, a video generation model that can create realistic videos from text. OpenAI describes Sora as a foundation for Artificial Intelligence (AI) that understands and simulates reality by training on publicly available resources online. In its launch statement, OpenAI also expressed the model's current limitations, as it still tends to generate unrealistic real-world conditions and struggles with complex actions over longer durations of generative videos.

OpenAI also states that all Sora-generated videos come with Coalition for Content Provenance and Authenticity (C2PA) metadata, which will identify a video produced from the Sora model for greater transparency. The metadata is also accompanied by visible watermarks that layer safeguards onto the generated video. Sora will also limit uploads of people at launch, blocking damaging forms of content, such as child sexual abuse materials and sexual deepfakes, until they can enhance deepfake mitigation solutions

Many Positives, but Also Some Avenues for Concern

IMPACT


OpenAI’s Sora is not the only text-to-video technology already in the market or development cycle, as other models like Shengshu Technology’s Vidu, Anthropic AI’s Pictory AI, and applications like Runway and Lightrick’s LTX Studio have made entryways into the emerging market. This technology, led by Gen AI’s strongest market leaders, could be the next catalyst for Large Language Models (LLMs) to take the next leap in unlocking new real-world applications and scenarios.

ABI Research expects Natural Language Processing (NLP) revenue to reach US$110 billion by 2030 at a 44% Compound Annual Growth Rate (CAGR). NLP’s potential is noted mainly in LLM’s ability to train and fine-tune its data to suit various needs, such as enterprise-grade insights. Having already witnessed how LLMs and Gen AI can change how we work and operate in a digitally transformed world, OpenAI’s Sora and many other similar technologies will further unlock myriad opportunities and use cases, such as:

  • Creative Direction: By converting text-based prompts into creative directions, animation directors or real-life films and moviemakers could enhance filmmaking in various ways, including integrating high levels of Computer-Generated Imagery (CGI) or simulating camera motion in film shots, among other uses. This could elevate the essential capabilities of storytelling through prompt-based creative direction, enhancing the director's and producer’s vision and imagination through Gen AI.
  • Influencer-Based Marketing: Natural language data collected on users can be used to prompt and craft personalized and targeted advertisements to a specific set of people or individuals, which generates better engagement potential between the product and the person.
  • Visual-Based Learning: Gen AI’s ability to craft personalized lesson tracks and plans for learning and curriculum has the potential to take this a step further, generating its content in bite-sized and digestible infographics and visual aids for learning. Further developments in short-form educational content development could be the next step toward personalized and engagement-driven quality education.

However, while the potential for Sora and its possible use cases continue to grow by the day, some critical avenues of concern must be considered. These include:

  • Looming Deepfake Dilemma: Even as OpenAI’s Sora has acknowledged the potentially harmful effects of deepfake and has taken direct steps to limit its potential, the potential for deepfakes to proliferate in other text-to-video LLMs is a key source of concern considering its socially destructive potential.
  • Social Engineered Attacks: Sora has demonstrated the ability to generate hyper-realistic videos, even coming close to simulating a live-like news broadcast. Without regulatory guidelines and key actions to mitigate such happenings, there is potential to proliferate fake news and simulated attacks on various fronts, including the social fabric of a country or at an enterprise’s level. Even with self-regulating measures, such as the C2PA metadata in place, potentially damaging societal impressions could have irreversible effects if disinformation is used and distributed unethically.

Urgent Need for Public Regulators to Define Trust and Safety

RECOMMENDATIONS


Given Gen AI’s growing adoption globally, Sora and OpenAI demonstrate the possibility of a market leader driving industry and self-regulation. For example, the C2PA is a collaboration between companies like OpenAI, Google, Publicis, Microsoft, and Adobe that are committed to mitigating the harms of synthetic or misleading content.

Technologies such as Gen AI have developed rapidly and exponentially in the last 4 years, leaving public regulators often playing catchup in regulatory measures. While its exponential growth could not have been predicted before the launch of OpenAI’s ChatGPT, regulators should acknowledge that Gen AI is here to stay and will likely define the growth of the global economy in the coming decade. Regulators must consider how they can stay ahead of the curve. Some recommendations by ABI Research include the following:

  • Fostering closer relationships with key AI leaders that could include them in understanding the innovation sector's development cycles and growth, allowing new AI regulations to be pre-emptively tabled for policy-based discussions and rolled out in tandem with new AI innovations.
  • Fostering inclusive policies that allow room for innovation, while structuring the regulatory road map for developers and innovators. This could be a key guiding strategy at a regional or national level that fosters an environment for ethical development, while ensuring strict regulatory standards, such as trust and safety, are met at every step of the development process.

Regulators are currently at a critical juncture in AI’s global development, and how they choose to move forward in the coming year will define their role in developing and regulating key public concerns like trust and safety for many years to come. Sora’s arrival for public use must be studied in the coming months, and its primary value proposition and underlying concerns should be carefully considered as more sophisticated AI developments, such as agentic AI and other newer innovations, are looming just beyond the horizon.

Services