Regulation's complexity exists on many levels. It is found in the crowded and contested regulatory space in which issues arise. This can make defining and diagnosing those issues problematic. Actors occupying the regulatory space compete to impose their frame on the issue, and to shape the manner in which the issue is defined and diagnosed, and regulatory objectives are set. The issues themselves are often embedded in complex social, economic and political systems that makes identifying their underlying causes hard; a difficulty compounded by the fact that other causes may be buried deep in the psychology and beliefs of those engaged in them. That regulatees have different attitudes, motivations and capabilities means there are no one-size-fits-all answers. This makes addressing regulatory issues and their causes all the more complicated. And whilst modern regulators have a vast arsenal of regulatory instruments and strategies at their disposal for tackling these issues - a testament to the creativity and ingenuity of the public sector - carefully selecting, combining, tailoring and aligning them to the specific nature of the issue and to regulatees' different attitudes and capabilities is inherently tricky. So too is designing and deploying the institutional structures and systems to support those endeavours. Ensuring they facilitate regulation in the public interest and guard against regulation in the private one is a constant challenge.

Navigating such complexities and challenges is a political exercise. The variables of regulatory policy - whether, what, and who to regulate, how, and to what standards - are, as Lasswell reminded us long ago, political decisions.[1] They involve balancing economic, social and democratic values, and the values and interests of different actors occupying the regulatory space, often producing both winners and losers. Moreover, these balances need to be struck in a manner that builds coalitions in support of the regulatory reform, and maintains those coalitions throughout the reform's life. Bad politics causes regulatory failure; good politics contributes to its success. 

The duality of regulation being both a policy reform and political project means it employs both technically rational and politically rational processes. Effective regulation involves the use of both systematic, analytical and evidence-based methods, as well as consensus and coalition-building consultation, negotiation and compromise. Of the two, however, politics tends to be the master; technical analysis the servant.[2] This is an important lesson. All those involved with the regulatory endeavour should be alert to efforts to hide politically contentious decisions in the neutral language with which most technical approaches are described. At the same time however, technical rationality should not be sacrificed on the altar of political expediency. Every effort should be made to place regulatory decision-making on a more technically rational foundation.

Regulation also is both a science and an art; and regulatory decision-making a craft. The science is evident in the many systematic and technical approaches championed by the better regulation movement. These include the use of stage-based policy cycles, regulatory impact assessments, responsive and risk-based regulatory strategies, and the many and varied techniques used to monitor and evaluate regulatory performance. We have also recently observed the increasing use of behavioural sciences to shape more effective regulatory tools. However, these systems, tools and technologies cannot (and should not) be robotically applied. Effective regulation requires the exercise of judgement and discretion. This is the artful part. Parameters need to be set; assumptions made; and hypotheses formed. The credibility and reliability of sometimes contradictory evidence needs to be assessed, and conflicting values and interests need to be weighed, balanced, and sometimes traded-off.

Exercising such discretion and making these judgements is a craft. Successful regulatory policy-making and implementation involves a mix of skills, knowledge and competencies - some technical and analytical; others relational and motivational; and yet others interpersonal and political. And while the craft can be studied, many of the traits required of a successful practitioner cannot be taught. As with most crafts, practice and experience is just as important as books and courses.

All of this makes governing through regulation inherently difficult. The potential causes of failure are many, as are the prescriptions for avoiding them and producing better and more legitimate regulation. Settling on the right combination of regulatory tools and strategies first time is likely to be the exception, not the rule.

Regulators (and their political overseers) need to foster a learning environment that enables regulatory practitioners to reflect upon and respond to lessons drawn from their own experiences and from those fof others. They also need to have a measured tolerance for mistakes. Perfection is an ideal - something to aspire to - not a benchmark against which regulators and regulatory regimes should be judged and held to account. Things will not always go to plan; and plans will not always be right. Mistakes will happen. What is important is that all involved in the regulatory endeavour learn - individually and organizationally - from them.

Regulatory scholars have a special responsibility in this regard. We have the luxury of time and distance that policy analysts and regulators immersed in day-to-day operations often do not have. Collectively we have developed a rich body of knowledge explaining the regulatory endeavour. It contains many valuable lessons, insights and perspectives. At the same time, however, it has remained remote from many practitioners.

As regulatory scholarship has become more sophisticated, it also has become increasingly complex and contested. For every theory there are one or more alternate or counter-theories, and numerous critiques. Moreover, many academics are more inclined to write for other academics than for policy-makers, regulators, practitioners and students of the field. Many of the typologies and classification systems they develop to explain and differentiate between aspects of the regulatory endeavour often employ labels and terms, and draw distinctions, that are not intuitive to the regulatory community. In this regard, it is not surprising that the theoretical advance to gain the most traction amongst them is Ayres' and Braithwaite's enforcement pyramid that is inductively based on the practice of regulators and expressed in language familiar to them.

The worlds of academia and policy-making have been described as two communities separated by different values, reward systems and languages.[3] The same can be said of the worlds of academia and regulatory practice. This divide needs to be bridged. Regulatory scholarship needs to be married with the real world of regulatory practice, and presented in the context of the practical challenges facing governments and regulators when deciding whether and how to regulate. It also needs to be presented to policy and regulatory practitioners in a manner that does not overwhelm by its complexity.*

The world in which we live is evolving at an exponential rate, creating new opportunities and ever more complex challenges. The conditions that initially saw regulation emerge as the expanding part of governance are intensifying, making governing through regulation the central task of the future. While there is unlikely to be a perfect regulatory regime, learning the lessons of the past and fostering the creativity of today is key to meeting the challenges of tomorrow.

* This is what I have sought to do in my forthcoming book: Governing through Regulation: Public Policy, Regulation and the Law (Routledge, August release: https://www.routledge.com/Governing-through-Regulation-Public-Policy-Regulation-and-the-Law/Eric/p/book/9781138935587). The book marries regulatory theory and practice into a holistic (and accessible) examination of the science and art of regulation.