Proponents
of a duty of care for online platforms have long dwelt on the theme of safety
by design. It has come to the fore again recently with the government’s
publication of a draft Statement of Strategic Priorities (SSP) for Ofcom under
the Online Safety Act. Safety by Design is
named as one of five key areas.
Ofcom is
required to have regard to the final version of the SSP in carrying out its
functions under the Act. Given Ofcom’s regulatory independence the government
can go only so far in suggesting how Ofcom should do its job. But, in effect, the
SSP gives Ofcom a heavy hint about the various directions in which the
government would like it to go.
So what does
safety by design really mean? How might it fit in with platform (U2U) and
search engine duties under the Online Safety Act (OSA)?
Before
delving into this, it is worth emphasising that although formulations of online
platform safety by design can range very widely [1] [2], for the purposes of
the OSA safety by design has to be viewed through the lens of the specific duties
imposed by the Act.
This piece
focuses on the Act’s U2U illegality duties. Three of the substantive duties
concern design or operation of the service:
- Preventing
users encountering priority illegal content by means of the service (S.
10(2)(a)) - Mitigating
and managing the risk of the service being used for the commission or
facilitation of a priority offence (as identified in the most recent illegal
content risk assessment of the service) (S. 10(2)(b)) - Mitigating
and managing the risks of physical or psychological harm to individuals (again as
identified in the most recent illegal content risk assessment) (S. 10(2)(c))
Two further
substantive illegality duties are operational, relating to:
- Minimising
the length of time for which priority illegal content is present on the service
(S. 10(3)(a)) - Swiftly
taking down illegal content where the service provider is alerted to or
otherwise becomes aware of its presence. (S. 10(3)(b))
S.10(4) of
the Act provides examples of the areas of design, operation and use of a
service to which the duties apply and, if proportionate, require the service
provider to take or use measures. Those include “design of functionalities,
algorithms and other features.”
Safety by
design in the Online Safety Act
When applied
to online speech, the notion of safety by design prompts some immediate
questions: What is safety? What is harm?
The OSA is
less than helpful about this. It does not define safety, or safety by design. It
defines harm as physical or psychological harm, but that term appears in only one
of the five substantive illegality duties outlined above. Harm has more a
pronounced, but not exclusive, place in the prior illegal content risk
assessment that a platform is required to undertake.
Safety by
design gained particular prominence with a last-minute House of Lords addition
to the Bill: an introductory ‘purpose’ clause. This amendment was the result of
cross-party collaboration between the then Conservative government and the
Labour Opposition.
What is now
Section 1 proclaims (among other things) that the Act provides for a new
regulatory framework which has the:
“general purpose of making the use of internet services
regulated by this Act safer for individuals in the United Kingdom.”
It goes on
that to achieve that purpose, the Act (among other things):
“imposes duties which, in broad terms, require providers of
services regulated by this Act to identify, mitigate and manage the risks of
harm (including risks which particularly affect individuals with a certain
characteristic) from
(i) illegal content and activity, and
(ii) content and activity that is
harmful to children, …”
Finally, and
most relevantly, it adds that:
“Duties imposed on providers by this Act seek to secure
(among other things) that services regulated by this Act are … safe by design…”.
A purpose
clause is intended to assist in the interpretation of the legislation by
setting out the purposes for Parliament intended to legislate, rather than
leaving the courts to infer them from the statutory language.
Whether such
clauses in fact tend to help or hinder is a matter of lawyerly debate. This
clause is especially confusing in its use of terms that are not defined by the
Act and do not have a clear and obvious ordinary meaning (“safe” and “safe by
design”), mixed up with terms that are specifically defined in the legislation
(“harm”, meaning physical or psychological harm).
One thought
might be that “safe” means safe from physical or psychological harm, and that “safe
by design” should be understood accordingly. However, that seems unlikely since
four of the five substantive illegality duties on service providers relate to
illegal content and activity per se, irrespective of whether they might
involve a risk of physical or psychological harm to individuals.
S.235 defines
Ofcom’s “online safety functions” in terms of all its functions under the Act. In
contrast, the transitional provisions for Video Service Providers define
“safety duties” in terms focused on platform duties in respect of illegality
and harm to children.
Similarly, in
the earlier part of the Act only those two sets of duties are described (albeit
merely in the section headings) as “safety duties”. “Safe by
design” may possibly refer to those duties alone.
The concept
of safety by design tends to embody some or all of a number of elements: risk-creating
features; prevention and reduction of harm; achieving those by appropriate
design of a risk-creating feature, or by adding technical safeguards.
The most
general aspect of safety by design concerns timing: that safety should
be designed in from the outset rather than thought about afterwards.
Prevention itself has a temporal aspect, but that
may relate as much to the kind of measure as to the stage of development at
which it should be considered. Thus the Minister’s introduction to the
Statement of Strategic Priorities says that it:
“includes ensuring safety is baked into platforms from the
start so more harm is caught before it occurs”.
This could
refer to the point at which a safety measure intervenes in the user’s activity, as opposed to (or as well as) the stage at which the designers consider it.
Later in the
Statement, safety by design is expressly said to include deploying technology
in content moderation processes. Providers would be expected to:
“…embed proportionate safety by design principles to mitigate
the [risk of their service being used to facilitate illegal activity]. This
should include steps such as … where proportionate, deploying technology to
improve the scale and effectiveness of content moderation, considering factors
including providers’ capacity and users’ freedom of expression and privacy
rights.”
An analogy
with product safety could suggest that safety by design is about identifying risk-creating
features at the design stage and either designing those features in the
safest way or incorporating safeguards. That aspect is emphasised by Professor
Lorna Woods in a recent paper [3]:
“The objective of ‘safety by design’ is – like product safety
– to reduce the tendency of a given feature or service to create or exacerbate
such issues.”
Applied to
products like cars that would mean that you should consider at the outset where
safely to position the fuel tank, not unthinkingly place it somewhere dangerous
and try to remedy the problem down the line, or after an accident has happened.
Or, if a piece of machinery has a sharp cutting blade, consider at the outset
how to add a guard into the design. A culture of safety by design should help
to ensure that potential safety risks are considered and not overlooked.
However, a focus on
risk-creating features gives rise to particular difficulties when safety by
design is translated to online platforms.
The underlying duty of care reasons for this have been rehearsed on previous occasions (here and here). In short, speech is not a tripping hazard, nor is
it a piece of machinery. A cutting machine that presents a risk of physical
injury to its operator is nothing like a space in which independent, sentient
human beings can converse with each other and choose what to say and do.
Professor
Woods [3] suggests that ‘by design’ seeks to ensure that products respect
the law (my emphasis). If that is right, then by the same token it could be
said that safety by design when applied to online platforms seeks to ensure
that in their communications with each other users respect the law (or
boundaries of harm set by the legislation). That is a materially different
exercise, for which analogies with product safety can be taken only so far.
The June
2021 DCMS/DSIT paper Principles of safer online platform design opened with
the statement that:
“Online harms can happen when features and functions on an
online platform create a risk to users’ safety.”
For the illegality
duties imposed by the OSA, when we set about identifying concrete features and
functionalities that are said to create or increase risk of illegality, we run
into problems when we move beyond positive platform conduct such as recommender
and curation algorithms.
The example of
recommender and curation algorithms has the merit of focusing on a feature that
the provider has designed and which can causally affect which user content is
provided to other users.
But the OSA
duties of care – and thus safety by design – go well beyond algorithmic social
media curation, extending to (for instance) platforms that do no more than
enable users to post to a plain vanilla discussion forum.
Consider the
OSA safety duties concerning priority illegal content and priority
offences. What kind of feature would create
or increase a risk of, for example, an online user deciding to offer boat trips
across the Channel to aspiring illegal immigrants?
The further
we move away from positive content-related functionality, the more difficult it
becomes to envisage how safety by design grounded in the notion of specific
risk-creating features and functions might map on to real-world technical features
of online platforms.
The draft SSP confirms that under the OSA safety by design is intended to be about more than
algorithms:
“When we discuss safety by design, we mean that regulated
providers should look at all areas of their services and business models,
including algorithms and functionalities, when considering how to protect all
users online. They should focus not only on managing risks but embedding safety
outcomes throughout the design and development of new features and
functionalities, and consider how to make existing features safer.”
Ofcom faced
the question of risk-creating features when preparing the risk profiles
that the Act requires it to provide for different kinds of in-scope service.
For the U2U illegality risk profile it has to:
“carry out risk assessments to identify and assess the
following risks of harm presented by [user to user] services of different
kinds—
(a) the risks of harm to individuals in
the United Kingdom presented by illegal content present on regulated
user-to-user services and by the use of such services for the commission or
facilitation of priority offences; …”
The risks
that Ofcom has to identify and assess, it should be noted, are not the bare
risk of illegal content or illegal activity, but the risk of harm (meaning
physical or psychological harm) to individuals presented by such content or
activity.
Ofcom is
required to identify characteristics of different kinds of services that are
relevant to those risks of harm, and to assess the impact of those kinds of
characteristics on such risks. “Characteristics” of a service include its
functionalities, user base, business model, governance and other systems and
processes.
Although a
platform has to carry out its own illegal content risk assessment, taking
account of Ofcom’s risk profile, the illegality risks that the platform has to
assess also include bare (non-harm-related) illegality.
Ofcom
recognises that functionalities are not necessarily risk-creating:
“Functionalities in general are not inherently positive
nor negative. They facilitate communication at scale and reduce frictions in
user-to-user interactions, making it possible to disseminate both positive and
harmful content. For example, users can engage with one another through direct
messaging and livestreaming, develop relationships and reduce social isolation.
In contrast, functionalities can also enable the sharing of illegal material
such as livestreams of terrorist atrocities or messages sent with the intent of
grooming children.” [6W.16]
Ofcom overcomes
this issue in its proposed risk profiles by going beyond characteristics that of
themselves create or increase risks of illegality. This is most clearly
expressed in Volume 2 of its Illegal Harms Consultation:
“We recognise that not all characteristics are inherently
harmful; we therefore use the term ‘risk factor’ to describe a
characteristic for which there is evidence of a risk of harm to individuals.
For example, a functionality like livestreaming is not inherently risky but
evidence has shown that it can be abused by perpetrators; when considering
specific offences such as terrorism or CSEA, a functionality like livestreaming
can give rise to risk of harm or the commission or facilitation of an offence.”
[5.26]
General
purpose functionality and features of online communication can thus be designated
as risk factors, on the basis that there is evidence that wrongdoers make use
of them or, in some instances, certain combinations of features.
Since
measures focused on general purpose features are likely to be vulnerable to
objections of disproportionate interference with freedom of expression, for
such features the focus of preventing or mitigating the identified risk is more
likely to be on other aspects of the platform’s design, on user options and
controls in relation to that feature (e.g. an option to disable the feature), or on
measures such as content moderation.
Ofcom implicitly
recognises this in the context of livestreaming:
“6.11 We acknowledge that some of the risk factors, which the
evidence has demonstrated are linked to a particular kind of illegal harm, can
also be beneficial to users. This can be in terms of the communication that
they facilitate, or in some cases fulfilling other objectives, such as protecting
user privacy. …
6.13 While livestreaming can be a risk factor for several
kinds of illegal harm as it can allow the real-time sharing of illegal content,
it also allows for real-time updates in news, providing crucial information to
a wide-range of individuals.
6.14 These considerations are a key part of the analysis underpinning
our Codes measures.”
The result
is that while the illegality risk profiles that Ofcom has proposed include as
risk factors a range of platform features that could be viewed as general
purpose, they tend not to translate into recommended measures aimed at
inhibiting that feature.
Here is a
selection of features included in the proposed illegality risk profile:
|
Service feature |
Risk (likelihood of increased risk of harm related |
|
Ability to create user profiles |
Grooming, harassment, stalking, threats, abuse, and for the risk of fake profiles: Grooming, harassment, stalking, threats, abuse, |
|
Users can form user groups |
Grooming, encouraging or assisting suicide or |
|
Livestreaming |
Terrorism, grooming, image-based CSAM, encouraging |
|
Direct messaging |
Grooming and CSAM, hate, harassment, stalking, |
|
Encrypted messaging |
Terrorism, grooming, CSAM, drugs and psychoactive |
|
Ability to comment on content |
Terrorism, grooming, encouraging or assisting |
|
Ability to post images or videos |
Terrorism, image-based CSAM, encouraging or |
|
Ability to repost or forward content |
Encouraging or assisting suicide or serious |
|
Ability to search for user generated content |
Drugs and psychoactive substances, firearms and |
|
Hyperlinks |
Terrorism, CSAM URLs, foreign interference offences. |
Other
functionality risk factors include anonymity, user connections (such as
friending and following), group messaging, and ability to post or send location
information.
Designation
of general purpose functionality as a risk factor reaches a high
point with hyperlinks. Since terrorists and other potential perpetrators can
use hyperlinks to point people to illegal material, hyperlinks can be designated
as a risk factor despite not being inherently harmful.
It is worth
recalling what the ECtHR said in Magyar Jeti Zrt (ECtHR) about the
central role of hyperlinks in internet communication:
“Furthermore, bearing in mind the
role of the Internet in enhancing the public’s access to news and information,
the Court points out that the very purpose of hyperlinks is, by directing to
other pages and web resources, to allow Internet users to navigate to and from
material in a network characterised by the availability of an immense amount of
information. Hyperlinks contribute to the smooth operation of the Internet by
making information accessible through linking it to each other.”
General purpose functionality as a risk factor was foreshadowed in the June
2021 DCMS paper. Arguably it went further, asserting in effect that providing a
platform for users to communicate with each other is itself a risk-creating
activity:
“Your users may be at increased risk
of online harms if your platform allows them to:
- interact
with each other, such as through chat, comments, liking or tagging - create
and share text, images, audio or video (user-generated content)”
In the
context of the internet in the 21st century, this list of features describes commonplace aspects of
the ability to communicate electronically. In a former age we might equally
have said that pen, paper, typewriter and the printing press are risk factors, since
perpetrators of wrongdoing may use written communications for their nefarious
purposes.
Whilst Ofcom
recognises the potential freedom of expression implications of treating general
purpose functionalities as illegality risk factors, it always has to be borne
in mind that from a fundamental rights perspective the starting point is that speech
is a right, not a risk. Indeed the Indian Supreme Court has held that the right
of freedom of expression includes the reach of online individual speech:
“There is no dispute that freedom of speech and expression includes the right to disseminate information to as wide a section of the population as is possible.”
That is not
to suggest that freedom of expression is an absolute right. But any
interference has to constitute a sufficiently clear and precise rule
(especially from the perspective of the user whose expression is liable to be
interfered with), then satisfy necessity and proportionality tests.
Preventative
technological measures
A
preventative approach to safety by design can easily lean towards technological
measures: since this is a technology product, technological preventative
measures should be designed in to the service and considered at the outset.
Professor
Woods [3], argues that:
“Designing for safety (or some other societal value) does not
equate to techno-solutionism (or techno-optimism); the reliance on a “magic
box” to solve society’s woes or provide a quick fix.”
However, in
the hands of government and regulators it has a strong tendency to do so.[4]. Indeed the draft SSP devotes one of its five
key priorities to Technology and Innovation, opening with:
“Technology is vital to protecting users online and for
platforms fulfilling their duties under the Act.”
Later:
“It is not enough that new, innovative solutions to known
problems exist – online service providers must also adopt and deploy these
solutions to improve user safety. … The government … encourages Ofcom to be
ambitious in its [code of practice] recommendations and ensure they maintain
pace with technology as it develops.”
We have
already seen that in the draft SSP, safety by design is said to include
deploying technology in content moderation processes.
On the basis
of prevention, an inbuilt technological design measure that reduces the amount
of (or exposure to) illegal user speech or activity should be preferable to
hiring legions of content moderators when the platform starts operating.
However, translating
duties of care or safety by design into automated or technology-assisted
content moderation can come into conflict with an approach in which
non-content-specific safety features are seen as preferable.
Professor
Woods said in the same paper:
“At the moment, content moderation seems to be in tension
with the design features that are influencing the creation of content in the
first place, making moderation a harder job. So, a “by design” approach is a
necessary precondition for ensuring that other ex post responses have a
chance of success.
While a “by design” approach is important, it is not
sufficient on its own; there will be a need to keep reviewing design choices
and updating them, as well as perhaps considering ex post measures to
deal with residual issues that cannot be designed out, even if the incidence of
such issues has been reduced.”
As to what ex
post measures might consist of, in a letter to The Times in August,
Professor Woods said:
“Through a duty of care, service operators are required to
ensure that their products are as safe as reasonably possible and to take steps
to mitigate unintended consequences. Essentially this is product safety, or
health and safety at work. This approach allows a range of interventions
that do not rely on content take-down and, indeed, could be content-neutral.
One example might be creator reward programmes that incentivise the spreading
of clickbait material. (emphasis added)].
Maeve Walsh,
writing for the Online Safety Network shortly before publication of the draft SSP [5],
contrasted safety by design with thinking about the OSA “primarily as a
takedown-focused regime, centering on individual pieces of content.”
Content-neutrality
suggests that a safety measure in relation to a functional feature should,
rather than relating specifically to some kind of illegal or harmful content,
either have no effect on content as such or, if it does affect user content, do
so agnostically.
Some
measures have no direct effect on user content: a help button would be an
example. Others may affect content, but are not targeted at particular kinds of
content: for instance, a friction-reducing measure like capping the permissible
number of reposts, or other measures inhibiting virality.
A measure
such as a quantititive cap on the use of some feature has the advantage from a rule of law perspective that
it can be clearly and precisely articulated. However, by virtue of the fact
that it constrains legitimate as well as illegitimate user speech across the
board, it is potentially vulnerable to proportionality objections.
Thanks to
the difficulty of making accurate illegality judgements, automated content
filtering and blocking technologies are potentially at risk on both scores.
[1] Trust
& Safety Professional Association. Safety by Design Curriculum chapter.
[2] Australian
eSafety Commissioner. Safety by Design.
[3] Professor
Lorna Woods, for the Online Safety Network (October 2024). Safety by Design.
[4] Maria P.
Angel, danah boyd (12 March 2024). Proceedings of 3rd ACM Computer Science and
Law Symposium (CSLAW’24) Techno-legal Solutionism: Regulating Children’s Online Safety in the United States.
[5] Maeve Walsh, for the Online Safety Network (11 October 2024). Safety by design: has its time finally come?
