https://bergstrom.org/tags/neural-networks/Neural Networks - Tag - Jorgen BergstromHugo -- gohugo.ioen-usJorgen BergstromSat, 26 Sep 2026 00:00:00 -0500https://bergstrom.org/posts/scalar_autograd_nn/Sat, 26 Sep 2026 00:00:00 -0500Jorgen Bergstromhttps://bergstrom.org/posts/scalar_autograd_nn/Introduction In the previous article we built engine.py: a complete reverse-mode automatic differentiation engine in about 80 lines, built around a single Value class. That engine knows how to record arithmetic on scalars and hand back the gradient of the final result with respect to every scalar that fed into it. This article builds the next layer on top: nn.py, a small neural-network library in roughly 60 lines (heres the repo).https://bergstrom.org/posts/cancer_prediction/Sat, 26 Sep 2026 00:00:00 -0500Jorgen Bergstromhttps://bergstrom.org/posts/cancer_prediction/Introduction In the first two articles of this series we built a reverse-mode autograd engine (engine.py) and a small neural-network library on top of it (nn.py). Together they are about 140 lines of pure Python with no numerical dependencies beyond the standard library. This article puts them to work on a real problem: predicting whether a breast tumor is malignant from measurements taken from a digitized image. The script is cancer_prediction.https://bergstrom.org/posts/backpropagation_part_3/Sat, 16 Nov 2024 00:00:00 -0500Jorgen Bergstromhttps://bergstrom.org/posts/backpropagation_part_3/[1 Input] + [1 Hidden Layer with N Perceptrons] + [1 Output] In this example I will extend my previous code to be able to handle a hidden layer with N perceptrons. As before, I will demonstrate the use of the neural network by fitting it to the following mathematical function: $y = 0.1 + 0.1 \cdot x^2$ over the range $x \in [0, 10]$. The C++ implementation for this example can be found in my github account.https://bergstrom.org/posts/backpropagation_part_2/Tue, 22 Oct 2024 00:00:00 -0500Jorgen Bergstromhttps://bergstrom.org/posts/backpropagation_part_2/https://bergstrom.org/posts/backpropagation_part_1/Sun, 20 Oct 2024 00:00:00 -0500Jorgen Bergstromhttps://bergstrom.org/posts/backpropagation_part_1/One Network Layer with 1 Input and 1 Output The simplest neural network architecture consists of a single perceptron. This network has only one input and one output, making it a highly streamlined model. While suitable for theoretical understanding, it is insufficient for real-world applications due to its limited capacity. However, it serves as a fundamental building block for studying the principles of backpropagation and training neural networks. Code Location You can find the source code for this example here.https://bergstrom.org/posts/nn_hammer_distance_pytorch/Wed, 29 May 2024 00:00:00 -0500Jorgen Bergstromhttps://bergstrom.org/posts/nn_hammer_distance_pytorch/Summary In this example I have used a “Sequential” neural network model to solve the regression problem of how far a hammer will fly given an initial velocity and angle. A sequential model is linear stack of layers, and allows you to create a Neural Network (NN) by simply adding layers sequentially. Both PyTorch and Keras are popular frameworks for building these NN models. Compared to Kears, the PyTorch approach is exposes more details which makes it more flexible and suitable for complex and dynamic models.https://bergstrom.org/posts/nn_hammer_distance_sklearn/Tue, 28 May 2024 00:00:00 -0500Jorgen Bergstromhttps://bergstrom.org/posts/nn_hammer_distance_sklearn/Summary The MLPRegressor in scikit-learn is a powerful tool for performing regression tasks using a multi-layer perceptron (MLP), which is a type of artificial neural network. It is a supervised learning algorithm that learns a function that maps input data to continuous output values. It can model complex relationships between features and the target variable. Code Location You can find the code in this example here. import matplotlib.pyplot as plt import numpy as np from sklearn.https://bergstrom.org/posts/nn_hammer_distance_tensorflow/Tue, 28 May 2024 00:00:00 -0500Jorgen Bergstromhttps://bergstrom.org/posts/nn_hammer_distance_tensorflow/Summary The Keras Sequential model is a simple and straightforward way to build neural networks in Keras, a high-level neural networks API running on top of TensorFlow. The Sequential model allows you to stack layers sequentially, meaning each layer has exactly one input tensor and one output tensor. You start by creating an instance of the Sequential model, then add layers to it one by one. Each layer, such as Dense (fully connected), Convolutional, or LSTM (Long Shor-Term Memory), is added using the add method.