Where to start building for long term AI safety and alignment?

/ Jonathan Hendler / Leave a comment

OpenAI and other AI companies will unlock a transition and the reorganization of nearly everything. Long term positive impact will be an increasing challenge, as we try to manage unintended consequences.

Short term safety issues are urgent, but clear as we work to manage malicious actors. There are broader international challenges, including with China.

While short term considerations are key, long term safety requires massive prediction and forecasting abilities, with the right initial conditions. What is an anchor AGI and superintelligence so that it doesn’t destroy us? The function may be one of freedom of consciousness for the most beings, as long as possible.

Via a prompt to GPT-4:

As we delve deeper into the realm of AI safety and alignment, it is crucial to explore technical examples of how one might construct an AI system with long-term alignment. Let’s consider the idea that the overarching function of AGI revolves around maximizing freedom of consciousness for the greatest number of beings, for as long as possible.

  1. Incorporating Value Alignment: To ensure the AI system aligns with the intended goal, we must explicitly integrate human values into the AGI’s objective function. This may involve techniques such as inverse reinforcement learning, where the AI system learns the preferences of human beings by observing their actions and decisions.
  2. Designing an Expandable Utility Function: To accommodate the evolving nature of human values and societal changes, the AI system’s utility function should be flexible and adaptive. It could be designed to incorporate new information, allowing the AI system to refine its understanding of the concept of “freedom of consciousness” and the best ways to maximize it.
  3. Incorporating Multi-Agent Collaboration: Since maximizing freedom of consciousness for the greatest number of beings might involve coordination among multiple AI systems, we should focus on building AI systems that can cooperate with other agents. Techniques such as multi-agent reinforcement learning can be employed to ensure that AI systems learn to collaborate effectively.
  4. Bias Mitigation: Ensuring fairness and avoiding biases in the AI system is crucial for maximizing freedom of consciousness. By incorporating techniques like adversarial training and fairness-aware machine learning, we can work towards building AI systems that treat all individuals fairly, regardless of their background or demographic characteristics.
  5. Long-term Impact Assessment: To evaluate the long-term consequences of the AI system’s actions, we can employ techniques such as Monte Carlo Tree Search (MCTS) and counterfactual simulations. These methods can help the AI system explore the potential outcomes of its decisions and choose actions that align with the goal of maximizing freedom of consciousness in the long run.
  6. Robustness and Safety Measures: To ensure the AI system remains aligned with the goal even under uncertain conditions, we can implement safety measures such as reward modeling, conservative policy updates, and distributional reinforcement learning. These techniques can help the AI system avoid catastrophic failures and maintain alignment with the objective of maximizing freedom of consciousness.

The next prompt:

Please expand on what “freedom of consciousness for the most beings, as long as possible” means. First, what is considered conscious, including definitions like Tsukumogami/animism and the belief that only some humans are. Also, what is freedom? Is it a metaphor for “freedom ends at the tip of my nose?” What kinds of ethical conflicts can arise when you try to include the most beings as possible? Finally what is the tradeoff between long term and short term efforts to preserve a consciousness?

The response:

“Freedom of consciousness for the most beings, as long as possible” is a multifaceted concept that entails addressing several key questions, such as the definition of consciousness, the meaning of freedom, potential ethical conflicts, and the trade-offs between short-term and long-term efforts.

  1. Defining Consciousness: The notion of consciousness is a subject of ongoing debate among philosophers, scientists, and AI researchers. In the context of AI alignment, we can consider a spectrum of definitions, ranging from the belief that only some humans possess consciousness, to more inclusive perspectives like animism or the concept of Tsukumogami (Japanese folklore, where inanimate objects can acquire a spirit after a certain period). Establishing a clear and precise definition of consciousness is essential for determining which beings should be considered when maximizing freedom of consciousness.
  2. Understanding Freedom: Freedom can be interpreted in various ways, often depending on cultural, philosophical, or individual perspectives. One common interpretation is the notion that “freedom ends at the tip of my nose,” meaning that an individual’s freedom should not infringe upon the rights or freedoms of others. In the context of AI alignment, freedom may involve ensuring that conscious beings can exercise their autonomy, make choices, and pursue their well-being without interference from AI systems or other external forces.
  3. Ethical Conflicts: When attempting to include the most beings possible, several ethical conflicts may arise. For instance, the interests of different beings may sometimes be at odds, leading to dilemmas where AI systems must prioritize certain beings over others. Additionally, cultural and philosophical differences in interpreting consciousness and freedom may complicate the decision-making process for AI systems. Balancing these competing interests and values is a significant challenge in AI alignment.
  4. Short-term vs. Long-term Efforts: Preserving consciousness might involve trade-offs between short-term and long-term efforts. Immediate actions may be necessary to protect conscious beings from immediate harm, while long-term efforts may focus on creating sustainable environments and societies that allow conscious beings to flourish. AI systems must balance these priorities, making decisions that may involve sacrificing short-term benefits for more significant long-term gains or vice versa. This balancing act may require advanced predictive capabilities and a deep understanding of the dynamic relationships between various factors that influence the well-being of conscious beings.

To design AI systems capable of maximizing freedom of consciousness for the most beings, as long as possible, it is crucial to address these complex questions and navigate the myriad ethical, philosophical, and practical challenges they present. By doing so, we can work towards creating AGI that not only aligns with our values but also respects the autonomy and well-being of all conscious beings.

By incorporating these technical approaches into the design and development of AI systems, we can work towards building AGI that genuinely aligns with the long-term goal of maximizing freedom of consciousness for the greatest number of beings, for as long as possible. This will require continuous research, collaboration, and innovation in the field of AI safety and alignment.




Measuring the digital divide

/ Jonathan Hendler / Leave a comment

How many calculations do I consume per day (or per second), compared to a person with less access to a smartphone or the Internet?

Computation per capita could be a scientific measure of the digital divide.

Computing may forever be bits and powered logic gates. We measure computer output with FLOPS. [1]  We can average total computational power per person. Segmenting by country adds more insight, which would be similar to maps of Internet access.

We make decisions in some parts of the world without much computation. To improve inequality we provide greater access to computation.

Google and other open services are a force for equality. The cost of a search for any user in one part of the world is near equal to all others. The equality isn’t perfect. Google has more data about me and how I live, so there’s more computation done on my behalf. But even if that is true, less Google is likely better than no Google.

We could try to measure computation per “decision”.

Decisions have measurable amounts of computation. What is the average amount of computation that happens when you tap your phone (in the cloud and on the phone)? I don’t know if we could measure the impact of a tap.[2]  But we could calculate the computational output of a click or tap.

How could you compare the list above to one hundred years ago, or one hundred years in the future?

Perhaps the closest analog to the Internet in the last century was the number of telegrams, pieces of mail, or minutes of phone calls;

  • to design and build a locomotive
  • buy a stock
  • buy a ticket on a trans-Atlantic zeppelin
  • decide when to plant corn
  • decide to get married

One hundred years in the future I’m not sure what we’ll be doing, but I hope we will still make useful decisions. Some possible examples:

What could we name “computations per decision”?

Since I’m an unqualified amateur, I’ll make up a term; decision density.

The number of bits used per a unit of time to make a perceived decision.

Note “perceived”. By definition, we must be able to observe when we’ve made a decision. We do so based on the support of many calculations, but most computation is hidden [3] and increasingly ubiquitous.

If we can measure decision density per human, we would have another tool to measure the digital divide.

I look forward to mathy, articulate, and design oriented humans to elevate my pre–post-human writing. Otherwise, I’ve got this post until I can ask Her to fix it.

Notes

Elon Musk mentions a “recursive y axis” in the video below as a way to measure progress, over time, of computation.

[1] – Factoring in energy you have performance per watt and Koomy’s law. This doesn’t tell us if the decisions were correct, or efficient (see big O notation or examples of bad code). Nor does this measure tell us how much pre-calculation went into a decision.

[2] – Maybe impact could be measured by how much money flows as a result. Or using HDI.

[3] – A calculation is contained in a function like f(x)=y, and the decision is the output. We could also say a calculation is made by a computer, and a decision is made by a human. (That line is already blurry).

Also – Check out the posts on Quora, the conversation on Hacker News

 

Starting up with Purpose

/ Jonathan Hendler / Leave a comment

It’s not news here in San Francisco,  in the most dominant region of the Silicon World – I’m creating a company. I have built a prototype. I’ve tested with users. I’m no miracle worker, and there’s a long, long way to go. And I’ve got just a couple of months to get off the ground.

What makes it worthwhile is the mission.

End the Digital Divide

The Digital Divide will end. The question is “Why?”. Even with rapid, global smartphone growth and internet from the sky, there’s so much important work to be done to learn, educate and protect.

If we work very very hard, and are very very lucky, only a minuscule part of the world will progress because of our efforts. That part matters. 

Starting a company, especially as a parent, isn’t for the faint of heart. I would be grateful for your feedback and support. Reach out to startup@hai.io, follow at @humanassisted or me @hendler

“Soon the digital divide will not be between the haves and the have-nots. It will be between the know-hows and the non-know-hows.” – Howard Rheingold

Sneak peek at early decisions at HAI

/ Jonathan Hendler / Leave a comment

Recently launched a site with some basic vision behind HAI.

http://hai.io

But the tech stack is where the rubber meets the road. I’ve been coding about two months now. At the very beginning I went through a fair amount of thinking and ended up selecting a language for the backend based on a number of factors. From languages I knew, C++, Go, PHP, Python, Java/Scala, and Node.js were on the table. Python and Java were the two top contenders, but I ended up going with Python. 

So far I’ve been really happy with Python for both flexibility of the language, the available libraries for both web and machine learning, and the developer community. Ruby / Rails has an amazing community and great web stack, but given my own lack of familiarity and less work being done in machine learning, it didn’t make my list.

Then I started evaluating open source projects that would be the platform. There are 132 on the list below (looked at least 4x that many). It’s been amazing getting up to speed on the projects that are open source. Although Google, IBM, Amazon and others are clearly going to lead in the machine learning space for the foreseeable future, the open source community is catching up.

Open source is a moving target, and there’s no one size fits all when you are piecing together something new. So, I’ve been using the awesome ZeroMQ library to connect services between libraries, languages.

Finally, thanks to everyone who has provided feedback so far. Can’t wait to get what I’m working on out into the world.

More about HAI

/ Jonathan Hendler / Leave a comment

For a few weeks I’ve been having meetings with advisors and colleagues. For those I’ve not been meeting, I’ve fallen behind in communicating what I’m working on.

Not a stealth startup, but there’s also a lot that’s yet to be determined. I’d prefer to be open, but there are some specifications that I’ll keep under wraps for a variety of reasons…. When building a stealth aircraft, at the least you can tell people that you are building an aircraft. Skunkworks doesn’t make sandwiches.

Company vision and culture will be in large part determined by cofounders. Here’s where there’s some definition:

  • HAI means AAI (Artificial Artificial Intelligence)  – humans intelligence built into a process that’s usable by computer intelligence
  • Ethical prime directives.  See Friendly AI
  • a sustainable business model early on
  • Company culture of sustainable innovation modeled after Google’s large revenue generating platform supporting R&D.
  • large, very talented, diverse founding team. Diversity is a no-brainer. Large is about five people; I’d rather create value from equity by distributing to founders than funders.
  • Boston still has untapped talent and potential. Even if developers can find jobs easily, what kind of job would an engineer want for the rest of their life?

Oh. HAI.

/ Jonathan Hendler / Leave a comment

HAI is the new company I’m working on. Human Assisted Intelligence. HAI will help computers learn about people. I’m so excited to be starting a new venture.

The future is happening right now. How far in the future will a product be relevant if you start developing it today?

Computer software / hardware outperforms humans in many specialized tasks today, and will likely surpass humans in categories reserved for our most revered public figures (scientists, politicians, performers) within 10-40 years. After the Singularity, quoting the WikiPedia main article -“Since the capabilities of such intelligence would be difficult for an unaided human mind to comprehend, the technological singularity is seen as an occurrence beyond which events cannot be predicted.”  Where will humans fit into this future? A question for science fiction, perhaps.

Before the Singularity, what will help us engage effectively with a world increasing its complexity, knowledge, and economic dynamics exponentially? If super computer intelligent systems are used only by the most powerful institutions, what kind of intelligent service represents the individual?

I think these are the most interesting, important challenges around AI. Today, I’m building a team and product prototypes. If you’re interested in collaborating, feel free reach out.

Earning leadership in parenthood and business.

/ Jonathan Hendler / Leave a comment

As a leader, control leaves your hands in one of two ways: by consent, or not by consent. Either way it happens. Choose consent.

Maybe this is the opposite of the Steve Jobs. I really don’t know, but feels like Jobs had a great sense of timing, how far he could fight for what he believed, and when to “give in”.

Perhaps there is no way to stop people from putting beans up their nose. Having insight ignored is painful if you care at all about what you are doing.  If the fight is too extended, no matter if right or wrong, you end up appearing arrogant or stubborn, but if you are right it doesn’t matter. And if a company needs focus and either way is right, it doesn’t matter which is chosen, only that something is chosen and committed to.

I read Herman Hesse’s “Journey to the East” twenty years ago. Summary: leadership is service. As a new parent, I think this is especially true.

Startups are built with both Undone and Unsung Engineering

/ Jonathan Hendler / Leave a comment
For me, web engineers are the magicians behind the curtain. That can mean not-a-lot of glory.

But for engineers like myself, undone engineering is more painful than unsung engineering.??One of the most difficult aspects of being a good engineer is knowing when to sacrifice good engineering. ??Startups, are by definition, challenging the established way of doing business, so the established way of doing great engineering sometimes isn’t good enough to survive. ??

( Although this is a topic pretty well covered in writing about startups , I try translate the constraints of a startup into what a priority list for technical leadership might look like. )??

Everything in a startup changes fast.??Engineering is managed chaos.??There are new team members, new customers, new data, new open source libraries, etc. Good engineering can accommodate change, but usually change is unexpected. As a side effect, the non-technical team begins to depend on the shortcuts, and the shortcuts begin to show their weaknesses. The design team has new ideas, and you can’t say yes as often as you used to.??

But you survive.??

Here are the priorities I use to keep everything moving for a web application startup (follows an 80 / 20 rule pretty well):

  1. protect the data (files/database, etc)
    1. back it up
    2. make sure data is valid (ideally some smoke tests, unit tests, regression tests on the data layer of an application)
    3. back it up
    4. source control with easy to understand policies and branching
    5. back it up
    6. automate backing it up
  2. fix problems with the user experiences
    1. small visual problems can make people … disgusted. Fonts, cross browser issues take a lot of time, but matter to people who expect things to work.??
    2. client (browser) speed – a small javascript issue can cause a page load to appear to take 5 seconds. Perceived performance is all that matters.??
    3. fix display issues – if they think data is lost, then it is??
    4. integrated QA team – make everyone on the team test the site, but not their own work.
  3. make coding fast
    1. establish team communication that makes product iteration rapid
    2. use frameworks, existing but stable libraries (open source so you can fix critical problems yourself)
    3. keep version control and deployment as automated as possible for the whole team
    4. write easy to understand code
    5. document code and best practices (wiki style)
    6. track problems (bugs and backlogs)
    7. write test code (on critical areas)
  4. housekeeping
    1. refactor
    2. cleanup
    3. more test code
  5. new features/ scaling/performance??- probably the most fun for an engineer, but last on the list. This has been the hardest for me. ??Engineers need to think about scaling when the system is architected…. maybe. But when you have just a 10 customers and the entire database fits in memory… why worry?
    1. database ??- a typical bottleneck. NoSQL and distributed architectures are making this easier
    2. separate web server from application layer. Concentrate core application logic into languages that are better suited for scaling and concurrency issues
    3. load balance, messaging queues, etc

Because of the list above, not every engineer out of a great tech school or amazing high tech corporation adjusts to startup life well. But when the above begins to pay dividends, engineers appreciate the big picture.

The divorce: Adobe Vs. Apple

/ Jonathan Hendler / Leave a comment

Apple and Adobe are both claiming a moral high-ground.

At first Adobe was the bad-guy for allegedly blocking further development of open standards like HTML5 and not opening up Flash.

Now, according to Adobe, Apple is the bad guy for not including flash and limiting development options for the iPhone (and iPad).
These are sad times because many are feeling like children of divorcees. Adobe and Apple have long provided shelter for creative geeks. Now the shelter is a crumbling, confusing world. Business and technology, especially when mixed, are sufficiently complex as to seem like magic. No magic here. Each company is changing, and getting over each other. Sorry lads. While tech-magic is love (translate: we will pay for magic, but nothing else) the magic eventually fades. The beginning of the fade is what I’ll call the magic line – the line between illusion, and disillusion. Adobe’s magic may be disappearing.
Is Adobe keeping up with demand for product improvements? Certainly, with software like Adobe’s, it is extremely difficult to sustainably write such complex software that runs the same, and well, on platforms you don’t control.
Perhaps, Apple has more power and responsibility than it realizes. As leaders in tech, Apple straddles the most thorny complex issues like copyright, DRM, hardware, open source; as both an under-dog and monopoly of tech.
There are reasons for the divorce, but they may not matter.

Open source, open platforms, and open standards are really the only sustainable way forward.

I say this not to take sides. As developers and creatives we must make choices that protect our future. Protect does not mean “defend”. Protect here means “ensures sustainable happiness for all involved”. Our destinies are already intertwined, but intertwined should not feel like a lock-in, trap or prison. Freedom can co-exist with interdependence.

At the least – businesses come and go, but openness wins.