The Smallest Brain You Can Build: A Perceptron in Python

comments (10)

• I think it should be quite obvious that perceptrons are far from the smallest units that are capable of learning. They store many bytes of information, require a non-local update process, need numeric (i.e. symbolic) inputs and involve relatively complex computations. You can go much simpler. For example:

  https://medium.com/@VictorBanev/the-simplest-learning-machin...

  This is a description of a 5-line algorithm that learns and stores approximate probability of an event using just 1 byte of persistent memory.

  romaniv 2d ago

• I think Karpathy's microgpt blogpost is the best in this genre in a long time, and it also includes a multi layer perceptron. It's a step up in the hierarchy, so reading both is helpful, of course.

  https://karpathy.github.io/2026/02/12/microgpt/

  kzrdude 2d ago

• In the early days of machine learning (before the first AI winter), networks like this were often implemented and trained in hardware: https://en.wikipedia.org/wiki/ADALINE

  That was the first thing that came to mind when I read "the smallest brain you can build". Nowadays, that "small brain" would likely be built on a breadboard using op-amps instead.

  rahen 2d ago

• If you want to learn the fundamentals of ML I recommend a book, such as Deep Learning: Foundations and Concepts by Chris Bishop. If you insist on staying online, one option is https://course.fast.ai/

  If you don't know ML I don't think you're going to learn much through ad hoc demos.

  esafak 3d ago

• The IF statement is the root creator of software programming. It has the ability to compare two values against each other and branch out to blocks of instructions. So it is perceiving (reading), decision making and routing - all that which differentiate life from inanimate objects. The AI agents perform the exact same loop, by delegating the first two steps to a model.

  Going further backwards, the transistor (or a PNP junction) is the hardware level enabler of the IF statement. The action (switching) driven by the current which in turn controls other switches, is the first manifestation of "observe and act" by inanimate things at the speed of electricity.

  Mechanical equivalents existed ofcourse - speed of a governer which controls the flow of fuel which in turn controls the speed of the governer.

  zkmon 2d ago

• One day I'll write about my 1-liner physics engine...

      let gravity = setInterval( _ => { if (projectile.object3D.position.y > 0) projectile.object3D.position.y \*= .99 }, 100)
  

  Jokes aside I find that providing ridiculously short toy examples that provide the very limited foundation of a concept are extremely empowering in pedagogy. You "get" it right away because it "fits" in your mind, then you dare tinker with it and quickly see how limited it is, then get excited again. It's a powerful trick to learn more IMHO.

  utopiah 2d ago

• I learnt a lot today from the interactive demo. You have the best clarity and right skill to educate

  ankit84 2d ago

• Nice and minimalistic

  I played with similar approach in JavaScript and built a NanoNeuron https://github.com/trekhleb/nano-neuron (it is more verbose than Python though)

  trekhleb 2d ago

• I wish that the tutorial went just one more step. It presents a one dimensional perceptron. But most perceptrons are multi-input. Adapting the article's 1D perceptron to three-input, for example:

      import random
  
      learning_rate = 0.1
      EPOCHS = 50
      NUM_INPUTS = 3
  
      weights = [random.uniform(-1, 1) for _ in range(NUM_INPUTS)]
      bias = random.uniform(-1, 1)
  
      data = []
      for _ in range(100):
          inputs = [random.uniform(-1, 1) for _ in range(NUM_INPUTS)]
          result = sum(inputs) > 0
          data.append((inputs, result))
  
      for epoch in range(EPOCHS):
          for inputs, result in data:
              weighted_sum = bias
              for i in range(NUM_INPUTS):
                  weighted_sum += inputs[i] * weights[i]
              
              prediction = weighted_sum > 0
              
              if prediction != result:
                  error = int(result) - int(prediction)
                  for i in range(NUM_INPUTS):
                      weights[i] += learning_rate * error * inputs[i]
                  bias += learning_rate * error
  
      print(f"Final weights: {[round(w, 3) for w in weights]}")
      print(f"Final bias: {round(bias, 3)}")

  SubiculumCode 2d ago

• Not a ML expert, but ML tutorials shall start with something like this... Good read. Thanks.

  virajk_31 2d ago