Cognitive World

Cognitive World is an experimental world simulation focused on agents that perceive their surroundings, retain knowledge, make decisions, and develop behavior through interaction rather than relying entirely on scripted routines.

Why I built it

Most game characters only appear intelligent because their behavior has been carefully predetermined. I wanted to explore what happens when agents are given limited perception, memory, goals, and the ability to update what they know about the world over time.

How it works

The planned system separates the simulated world from each agent’s internal understanding of it. Agents receive incomplete observations, store selected memories, evaluate goals, and choose actions based on what they currently believe. Deterministic systems manage world state and physical rules, while AI components can be used where interpretation or flexible planning adds value.

Currently the system I work in primarily to test this and further development is a combination of Python based orchestration of AI and deterministic systems with a Java framework. A perfect playground for this would be one that makes most external needs to the goal simple. As a computer fanatic, I grew up playing Minecraft and using the mods on their and i was always fascinated by the modding communities ability to change the game world while still holding the principles of interaction design in tact. So I decided minecraft and Java would provide a 3D engine base to test integrations from the Python orchestration layer with a low level Java layer in a game world.

The python service is planned to operate as the executive or head in the system. Data comes into it and it spits out updated state, new values, and changes to the world that is percieved. The goal is for this to happen as close to real time as possible. The local game or integrated system saves and maintains state. Keeping state maintained at this lower level prioritizes deterministic game data over inferenced data generated or output by the python orchestration layer.

Initial designs have a plan like this:

• The game starts out by the player or user inputting a story line, a tone, or a narrative style they enjoy into the Python Executive/Orchestration layer. This in turn runs a multi-agent job to generate a story line, agent personalities, and more in accordance with the integrated system - so far this would be minecraft. These are taken as initial values for the future flow of the game.
• As the player interacts with the world the lower level system in Java that functions within the game world updates local state such as position values, world or environment changes, or real-time interactions between agents and the player. These are communicated with the Python orchestration layer which then in turn would respond to interactions or make decisions based on the incoming state.

What I’d do differently

The idea can become too broad very quickly. I would begin with one small environment, a limited set of needs, and measurable agent behaviors before adding language models, complex societies, or larger world systems. Currently I have split the development in two, what I call the “Orchestration Layer” named CognitiveWorld (Python service) and the actual in game integration of the CognitiveWorld system, which is currently a Java mod in the minecraft modding framework called CognitiveBlocks. It started out as CognitiveBlocks then turned into the two seperate systems. Their are many personal ambitions behind the project but one of the biggest things I would change from the beginning is honestly starting smaller and working into the long term goal. Which is a skill I believe I am beginning to learn as an Engineer.