AI coding agents have become an essential part of a modern developer’s workflow. But have you ever felt unwell about sending your proprietary code, company secrets, or sensitive customer data to external cloud servers?

Understandable. More and more companies are limiting the use of cloud-based AI for exactly these data privacy reasons. What if you could get the same—or even better—performance completely free and running locally on your computer?

There’s a way to run an extremely powerful AI coding setup right on your local machine—completely offline, no account required, and fully GDPR-compliant.

Whether you’re on a train, on a plane, or in the middle of nowhere with no signal, your coding AI always runs at full power with zero lag from bad internet.

In this guide, we’ll show you how to set up Continue.dev in VS Code and how we use it in our development workflow on our MacBook Pro M4 Max with 48 GB of Unified Memory.

By the way: Our productivity has skyrocketed.

We’ll discuss the following points:

• What Is Continue.dev?

• Top Features of Continue.dev

• Step-by-Step Guide: Our 100% Local Setup

• How the M4 Max Performs in Everyday Use

• Cloud vs. Local: The Direct Comparison

• Conclusion: The Era of Cloud-Dependent AI Is Over

💡 PLUS: Our exact config.yaml configuration.

Continue reading on Steady

AI agents offer a unique opportunity to automate and enhance complex processes. But to maximize the full potential of agentic AI, it is crucial that AI agents are able to collaborate within a dynamic multi-agent ecosystem.

Agents need to be able to work together seamlessly, even if they were developed by different vendors or built on different frameworks. This increases the autonomy of these systems and reduces costs in the long run.

This is where the Agent2Agent (A2A) protocol comes into play. It’s an open standard designed for seamless communication and collaboration between AI agents.

This guide introduces the fundamentals of the A2A protocol and demonstrates how to use it with the Agno framework.

We’ll discuss the following points:

• What is Agent2Agent (A2A) Protocol?

  • Problems that A2A Solves

  • Key Concepts and Components

  • How A2A and MCP Work Together

• Technical Requirements

• Expose Agno Agents via the A2A Protocol

• Connect an A2A Client to A2A Servers

• Conclusion

Continue reading on Steady

AI agents are becoming increasingly autonomous and complex. The trend is moving toward Agentic AI—systems that not only respond but also take independent actions, plan tasks, and interact with other agents.

This requires a platform that reliably manages this autonomy. The solution is an AgentOS, which orchestrates, monitors, and controls these agents.

Agno’s AgentOS is a production-ready runtime that can run entirely within your own infrastructure. It serves as a central hub for your AI agents and provides a unified API for execution, monitoring, and management.

In this detailed guide, we show you how to expose an AgentOS as an MCP server for external clients that can handle MCP-compatible applications.

We’ll discuss the following points:

• Why Use MCP (Model Context Protocol)

• Technical Requirements

• Implementation: AgentOS as an MCP Server

  • Set Up the Database With Docker

  • Create the Agent Logic Using Ollama

  • Set Up the AgentOS With MCP Enabled

  • Overview of Available MCP Tools

• Implementation of an MCP Client

• Testing AgentOS MCP

• Conclusion

Continue reading on Steady

Updated: May 30, 2026

Our AI Engineering Hub is a guide-based learning platform. You can work through each guide independently.

Discover our comprehensive collection of practical guides covering various topics such as Generative AI, Large Language Models, Agentic AI, Python in Finance, Machine Learning methods, and more. Enjoy learning!

Become a member

😉 Insights

• 🆓 Our Journey to Become a Data Scientist

• 🆓 DSBoost Interview

🐍 Python in Finance

• 🆓 A Beginner’s Guide to the OpenBB

• 🔒 EDGAR Tools: Python Package to Get SEC Filings

• 🆓 Bollinger Bands: A Guide with Python and OpenBB

• 🔒 DateTimeIndex in Pandas for Time Series Analysis

• 🔒 Stock Market Data: Time Resampling with OpenBB and Pandas

• 🆓 Time Shifting in Pandas Using Stock Market Data

• 🔒 Portfolio Allocation: Analyze a Stock Portfolio Using Python

• 🔒 Mastering the Capital Asset Pricing Model (CAPM) Using Python

• 🔒 Portfolio Optimization Using the Markowitz Model in Python

• 🔒 Build a Segmented Revenues API for Public Companies

• 🔒 Visualize financial data with beautiful Sankey diagrams

• 🔒 Convert a Time Series into a Classification Problem

• 🔒 Detection of Credit Card Fraud with an Autoencoder

🧠 Machine Learning Methods

• 🔒 Linear Regression: Theory and Practice

• 🔒 A Visual Guide to Ensemble Methods + Practical Example

• 🆓 Artificial Neural Network from Scratch

• 🆓 Convolutional Neural Networks: A Beginner Guide

• 🔒 Basic Concepts Of Explainable Artificial Intelligence

📈 Statistics

• 🔒 Univariate Discrete Distributions

• 🆓 Bootstrap Method: An Introduction With a Python Example

• 🆓 An Introduction to Cross-Validation

• 🆓 Return Correlation in the Portfolio Example

• 🔒 Time Series Analysis: A Beginner’s Guide with statsmodels

• 🔒 Introduction to ARIMA Models for Newbies

• 🔒 An Introduction to ETS Models

• 🔒 Time Series Analysis: Exponentially Weighted Moving Average

🤖 GenAI, LLMs, and RAG

• 🆓 Run GenAI Models locally with Docker Model Runner

• 🆓 Build a Local RAG App to Chat with Earnings Reports

• 🆓 Build a Local Chatbot in Minutes with Chainlit

• 🔒 LangChain: A Beginner Guide

• 🔒 LangChain: Chat with Earnings Reports of Companies

• 🔒 LangChain: Chat with an Earnings Call on YouTube

• 🔒 Prompt Engineering for Developers: A Practical Guide

• 🔒 Build an OpenAI-Compatible API for Open-Source LLMs from Scratch

• 🔒 Responsible Development of an LLM Application

• 🔒 Keyword Search and LLMs

• 🔒 Lamini: Fine-Tune a Large Language Model

↔️ Agent Protocol Stack

• 🆓 Anthropic’s Model Context Protocol (MCP) with Python

• 🆓 Expose an Agno Agent as an AG-UI Compatible App

• 🔒 Turn Your Agno AgentOS Into a Powerful MCP Server

• 🔒 How to Expose Agno Agents Using the A2A Protocol

✨ Agentic AI Systems

• 🆓 Agent Engineering with Agno

• 🆓 Build an AI Finance Agent Team

• 🆓 Build an AI Investment Agent with Chainlit

• 🆓 Build a Local AI Agent to Chat with Financial Charts Using Agno

• 🆓 Build a Local AI Agent with Memory Using Agno

• 🆓 Build a Local AI Agent with Knowledge and Storage Using Agno

• 🆓 Build a Multi-Agent Stock Market Analyst to Compare Stock Price Performance

• 🆓 Create a Local AgentOS with Agno

• 🔒 Turn Your Agno AgentOS Into a Powerful MCP Server

• 🔒 Build a financial multi-agent system with AG2 and Ollama

👨🏽‍💻 Programming Best Practices

• 🆓 7 Hidden Python Tips

• 🆓 Speed Up your NumPy Code with NumExpr

• 🆓 Tips and Tricks to Improve Your R Skills

• 🆓 Unlock the Full Potential of Python

• 🆓 Speed Up your Python Skills

🖥️ Web Development

• 🆓 REST API: The Hero of Modern Web Development

• 🆓 An Introduction to FastAPI

• 🆓 Understand Service Workers of a PWA (Part 1)

• 🆓 Understand the Offline Functionality of a PWA (Part 2)

• 🔒 Build a Chatbot with Plotly Dash

• 🔒 Build Enterprise-Level Apps with Plotly Dash

• 🆓 Build Web Apps with Reflex in Minutes Using Pure Python

• 🔒 Build a Stock Dashboard with Reflex and OpenBB

• 🔒 Build Full-Stack Chatbot App in Python Using Reflex and Agno

• 🔒 Deploy a Plotly Dash App to Hugging Face Using Docker

• 🆓 Deploy a Web App With Docker on Render

📊 Data Visualization

• 🆓 Create Impressive Charts Using Plotly Express in Python

• 🔒 Visualize financial data with beautiful Sankey diagrams

💻 Development and Operations

• 🆓 Homebrew - Package Manager for Mac

• 🆓 Using Miniconda x86_64 & Apple Silicon Side-by-Side

• 🆓 A beginner-friendly introduction to Docker

• 🆓 Create a local AI Coding Assistant for VSCode

• 🔒 Our Ultimate Local AI Coding Setup in VS Code

• 🔒 Poetry: Python Dependency Management

• 🔒 Pytest: Efficient Testing of Python Code

• 🆓 Set Up a PostgreSQL Database With Docker

• 🆓 Working with Time Series Data Using Postgres

• 🆓 Set Up a GitLab Server with Docker

• 🔒 Set up a Docker-based SFTP server

• 🔒 MLflow Workspace On-Premise with Docker

📱 Project Management

• 🆓 Code Documentation with MkDocs

• 🆓 CRISP-DM: CRoss Industry Standard Process for Data Mining

• 🔒 Efficient Use of KPIs in Data Science Projects

In a world full of AI agents, data is the most valuable asset. That’s the reason why it’s essential to have access to high-quality data. But how do you get access to this data? With an API.

You can access an API either by paying for it or by creating your own. Most APIs are expensive and often fail to provide all the necessary data for a specific application, particularly in the investment industry.

A custom API lets you customize data sources, formats, and features to meet your exact needs. You can integrate specific financial data (e.g., stock prices, balance sheet data, segmented revenues) that are relevant to your application or analysis.

In addition, you can adapt your API flexibly to new requirements, such as integrating new markets or asset classes. The mission of this article is to build a Segmented Revenues API based on the official company reports.

We’ll discuss the following points:

• Technical requirements

• Business understanding and data modeling

  • Revenue segmentation logic and structure

  • Entity-relationship diagram

• Setting up the database with Docker

  • Adminer and TimescaleDB

  • Overview of the Docker stack

  • Creating and configuring the database schema

• Data preparation

  • Downloading and cleaning 10-K filings

  • Inserting data into the database

• Building the Segmented Revenues API with FastAPI

  • Set up a basic FastAPI endpoint

  • Designing the API endpoint

  • Querying segmented revenue data

• Testing the API

  • Sample request

  • Creating a Python client

• Conclusion

So we’ve no time to waste. Let’s build a segmented revenues API from scratch! 🚀

Continue reading on Steady

Have you ever wondered if you can build production-ready analytics apps in pure Python? Then, the open-source framework Reflex might be perfect for you!

With Reflex, you can build the frontend and backend of your app in Python - no Javascript required. It supports over 60+ built-in components and allows you to wrap any React component. Additionally, everything is fully responsive, ensuring your app looks great on any device. 

In this step-by-step guide, you’ll learn how to build a stunning stock dashboard app using real-world data with the Python package openbb. In addition, we’ll show you how to structure the app effectively to keep it easy to maintain over time.

Sneak Peak 👇🏽

We’ll discuss the following points:

• Technical requirements

• Initialization of the app

  • App Initial Setup

  • Project Structure

  • Styles for the App

• App Components

  • Card

  • Navbar

  • Sidebar

• Models

• App Pages

  • Page Template

  • Dashboard

  • Settings

• Conclusion

So we’ve no time to waste. Let’s build a beautiful-looking stock dashboard from scratch! 🚀

Continue reading on Steady

More and more companies are integrating intelligent chatbots into their apps and websites. The advantage of chatbots is that they are available 24/7. That allows companies to save time and money.

In the next few years, almost every app will have a chat user interface. Most people are already familiar with this interface from apps like WhatsApp. Meta, Google, and Microsoft have already begun integrating intelligent chat interfaces into their products.

The intelligent chatbots of the future will not only be able to output text but also offer widgets and other interaction options. For this reason, it makes sense to learn how to create custom chat interfaces.

In this guide, we’ll explain how to build a well-structured chat app using Reflex. We’ll also use Agno as a multi-agent framework.

Sneak Peak 👇🏽

We’ll discuss the following points:

• Why does a custom chat interface make sense?

• Tech Stack

  • Web App Framework Reflex

  • Multi-Agent Framework Agno

• Model View Controller Pattern

  • Advantages

  • Disadvantages

• Setup of the development environment

• Best Practices: Project Structure of the Chat App

• Implementation of the Chat App

• Conclusion

So we’ve no time to waste. Let’s build a full-stack chatbot app in pure Python from scratch! 🚀

Continue reading on Steady

The analysis of historical financial market data can reveal interesting patterns and insights. This information can help to build a strong portfolio. When creating a portfolio, two key variables play a significant role: diversification and the risk-return ratio.

The diversification of a portfolio can be easily implemented by adding different assets from various sectors. However, many retail investors overlook the evaluation of the risk-return ratio.

In this guide, we will explain how to apply and use the Capital Asset Pricing Model (CAPM). The CAPM is a theoretical model used to analyze the risk and return of securities. The model’s goal is to determine expected returns based on the risk involved.

We will discuss the following points:

• Introduction to the Capital Asset Pricing Model

• Difference between systematic and unsystematic risk

• CAPM Calculation with a simple example

• Advantages and disadvantages of the CAPM

• Example with Python

  • Technical requirements

  • Download stock market data with OpenBB

  • Exploratory analysis

  • Application of the CAPM

• Conclusion

Continue reading on Steady

Portfolio optimization focuses on selecting the optimal combination of assets to maximize return and minimize risk. That involves determining the most appropriate weights for each asset.

The historical performance of the assets, their correlations with each other, and other relevant factors are considered. The goal is to create a well-diversified portfolio with a balanced relationship between risk and return.

In this guide, you’ll learn how portfolio optimization works using the Markowitz model with the Sharpe Ratio. We’ve already explained the Sharpe Ratio in detail in another article, which we highly recommend you read first.

Here’s an overview of the topics:

• How do you optimize a portfolio?

• Data access and creating a demo portfolio

• Technical requirements

• Metrics and Exploratory Data Analysis

• Portfolio Optimization from Scratch

• Portfolio Optimization using SciPy

• Calculate the Efficient Frontier

• Portfolio performance against a benchmark

• Conclusion

Let’s get started to find the optimal portfolio allocation using the Markowitz Model. 😃

Continue reading on Steady

Many retail investors worldwide invest their money in the stock market and build a stock portfolio. Wouldn’t it be helpful if you could create a risk-adjusted portfolio? In other words, more return with less risk!

Sounds this good for you. Then this article is right for you. We examine a stock portfolio with Python and calculate some important portfolio statistics, including the popular Sharpe Ratio. In this article, you’ll learn how to analyze your portfolio in terms of risk.

We’ll discuss the following points:

• Technical requirements

• What is a Stock Portfolio?

• Portfolio Statistics

• What is the Sharpe Ratio?

• Evaluation of a Portfolio with Python

  • Load Stock Data with OpenBB

  • Portfolio Allocation

  • Portfolio Performance

  • Risk Evaluation

• Conclusion

Continue reading on Steady

New and powerful open-source AI models are entering the market every few months. DeepSeek R1 was released a few weeks ago, and the AI world was excited. According to DeepSeek, the model was much cheaper to train and performed better than OpenAI’s o1 model in several benchmarks.

Open-source models like DeepSeek or Llama 3 provide many benefits for developers. For example, you can fine-tune them on-premise to create large language models (LLMs) for your specific use cases. However, there’s also the challenge of how to combine the strengths of different models in one application efficiently. The solution is to build your own REST API.

With a REST API, you can host various open-source models customized for your applications. So, you can use the different strengths of each model. In addition, an own API is more cost-efficient in the long term and ensures data privacy.

Many Python libraries support the API schema of OpenAI. For this reason, it makes sense to build an OpenAI-compatible API. To do this, we’ve followed the official OpenAI API reference.

We’ll discuss the following points:

• Benefits of an own API

• Technical requirements

• Create a Chat Completion API

  • Simple REST API with FastAPI

  • Q&A Endpoint without Streaming

  • Q&A Endpoint with Streaming

  • Tool Use Support

• Example: Web Search Tool with AG2

  • Configuration

  • Definition of the Tool

  • Definition of the Agents

  • Registration of the Agents

  • Testing the Web Search Tool

• Conclusion

• Appendix: Full API Code

Continue reading on Steady

The analysis of financial data can quickly become very complex. This is where smart data visualization is helpful. It makes understanding the data easier.

Sankey diagrams are a special type of flowchart, typically used to visualize the flow of measurable resources. They draw the reader’s attention because they explain complex data clearly. An intelligent choice of colors allows the reader to understand different flow quantities intuitively.

In this guide, we will show you how to create beautiful Sankey diagrams using Plotly. These diagrams are often used in the financial industry and are very helpful for visualizing financial statements.

We guide you through the entire process, from retrieving data to creating impressive Sankey diagrams. This way, you can create Sankey diagrams like the one shown in the example below.

We will discuss the following points:

• What is a Sankey diagram?

  • Key Terms

  • Strengths and weaknesses of Sankey diagrams

  • Use Cases

• Technical requirements

• Interactive Sankey diagrams with Plotly

• Create a Sankey diagram of Apple's income statement

  • 10-K filing: Data Access in Python

  • Extract Apple’s financial data from the 10-K filing

  • Sankey diagram “How Apple Makes Money?“

• Conclusion

Continue reading on Steady

The big tech companies say the future belongs to AI agents. And we think the same. Multi-agent collaboration will soon be an essential part of every software.

Microsoft recently announced a swarm of new agents for 2025 in its products at the Ignite conference. As data scientists and software engineers, it is essential to understand how AI agents work and how you can use them efficiently.

In our view, the companies of the future will be driven by AI agents, which will give these companies an economic advantage over other companies. Therefore, it's time to take a closer look at multi-agent systems.

But what is the best way to start? 🤔

We have spent months with several frameworks for multi-agent systems, and we struggle with the complexity of it all. Then we came across AG2 (formerly AutoGen), and it offers exactly what we need.

AG2 offers features like conversation patterns, support for tool use, and human-in-the-loop workflows. In short, this framework simplifies the process of developing production-ready multi-agent applications.

In this article, we build a multi-agent system to perform a financial analysis of the Tesla stock. Best of all, we run it locally, so you don't have any API costs.

We’ll discuss the following points:

• What is AG2?

• Technical requirements

• Agents in AG2

  • Conversable Agent

  • Assistant Agent

  • User Proxy Agent

  • Group Chat Manager

• What are Conversation Patterns?

  • Two-Agent Chat

  • Sequential Chat

  • Group Chat

  • Nested Chat

• What are Agentic Design Patterns?

  • Reflection

  • Tool Use

  • Planning

  • Multi-Agent Collaboration

• Perform a multi-agent financial analysis of Tesla stock

  • Configuration

  • Definition of the Task

  • Definition of the Agents

  • Build a Group Chat

• Conclusion

So we’ve no time to waste. Let’s dive in!

What is AG2?

When you are searching for "AutoGen" online, you might feel confused at first. There are two frameworks that are related to AutoGen: AG2 (formerly AutoGen) and Microsoft’s AutoGen.

AG2 continues the original AutoGen framework and is maintained by its original creators. The original creators left Microsoft and rebranded the framework. It is now hosted under the "AG2ai" organization. AG2 focuses on a community-driven approach with open governance.

Microsoft continues to develop the framework AutoGen independently of AG2. In addition, Microsoft has introduced major updates, including a complete rewrite of version 0.4. In this context, Microsoft wants to adapt the framework so it better fits Microsoft programming interfaces.

In this article, we use the framework AG2. It is an open-source framework designed for building AI agents and enabling multi-agent collaboration to solve tasks. The framework simplifies the orchestration and automation of complex LLM workflow. It’s like PyTorch for building AI agents.

Continue reading on Steady

ARIMA (short for AutoRegressive Integrated Moving Average) is a technique for time series analysis and forecasting. It is one of the most widely used approaches for time series forecasts. There are seasonal and non-seasonal ARIMA models, depending on the type of time series data.

ARIMA models are often used in the financial industry. They are fundamental to understanding time series analysis. In addition, ARIMA models can be very complex, which means that they are not always easy to understand.

In this tutorial, we would like to give you an easy-to-understand and visual introduction to ARIMA models.

We’ll discuss the following topics:

• Introduction to ARIMA

  • Components of an ARIMA model

  • Stationary vs. Non-Stationary

  • Differentiation

• Autocorrelation Plots

• Practical example: ARIMA with statsmodels

  • Technical requirements

  • Use Case and Dataset

  • Time series visualization

  • Stationarity check

  • Autocorrelation Plots

  • Create ARIMA model

  • Forecasting with ARIMA

• Conclusion

Continue reading on Steady

The Exponentially Weighted Moving Average (EWMA) is a statistical tool for modeling or describing a time series. It is commonly used in the financial industry to evaluate the risk of securities or portfolios.

In this article, we will introduce you to the mathematical basics of the EWMA. We will also show you how to use EWMA models in Python using practical examples.

Understanding how EWMA works is essential for a data scientist or quantitative analyst, as it is widely used in the financial industry. You can immediately apply the knowledge from this article in your daily work.

Here’s an overview of the topics:

• What is the Exponentially Weighted Moving Average (EWMA)?

• Applications of the EWMA

  • Technical Analysis

  • Risk Management

• Practical examples with Python

  • Technical requirements

  • Use Case and Dataset

  • Weaknesses of the Simple Moving Average (SMA)

  • Exponentially Weighted Moving Average

• Conclusion

Continue reading on Steady

Python is one of the most popular programming languages for applied finance and machine learning. In the financial sector, a lot of data is available as time series.

For data scientists or quant scientists, it is therefore essential to master time series analysis techniques. Time series data have unique properties and need different algorithms.

We have written an introduction article on time series analysis. In this article, you learn everything you need to know about time series analysis with statsmodels.

There are several time series models, but in this tutorial, we will focus on ETS models. As a data scientist in the financial sector, it is essential to know these models.

We’ll discuss the following topics:

• Theory of Error-Trend-Seasonality (ETS) models

• Technical requirements

• Practical Use Case

  • Description of the Use Case

  • Working with a financial API in Python

  • Data Preparation

  • Data Visualization

• ETS models with Python

  • Time Series decomposition

  • Simple exponential smoothing

  • Holt-Winters Method

• Conclusion

Continue reading on Steady

Last updated: September 23, 2024

Time series analysis is a powerful statistical method used in many industries, such as finance, healthcare, energy, and marketing. It involves identifying patterns, trends, seasonality, and irregularities in data observed over different time periods. These methods are useful for understanding the underlying structure and patterns in the data.

In this guide, we will go into the details of time series analysis. In addition to the basic concept, we introduce popular time series techniques. We also demonstrate how to perform a time series analysis with a practical example.

As a data scientist, it's essential to dive into the world of time series data! Our guide will take you on this exciting journey.

We’ll discuss the following points:

• What is Time Series Analysis?

• Time Series Basics

  • Elements of Time Series Data

  • Important Time Series Concepts & Terms

• Time Series Analysis Methods

• Limitations of Time Series Analysis

• Use Cases for Time Series Analysis

• Introduction to statsmodels

  • Setup and technical requirements

  • Practical Example with economic data

• Conclusion

Continue reading on Steady

Ensemble methods are a powerful approach in machine learning that combines multiple models to create a more robust and accurate model. Many beginners struggle to understand how ensemble methods work, so we want to explain the concepts behind ensemble methods visually. No math today!

The primary idea behind ensemble methods is that the aggregation of multiple models helps reduce the weaknesses of each model.

This assumption is based on the idea that each model has its own strengths. By combining them, we can create a more complete and reliable model.

As a Data Scientist, it is essential to understand how the basic strategies behind ensemble methods work.

We’ll discuss the following points:

• Motivation behind it

• Strategies for ensembles

  • Boosting - How it works?

  • Bagging - How it works?

  • Stacking - How it works?

• Real-world example with stock market data

  • Technical requirements

  • Business Understanding

  • Data Understanding and Preparation

  • Modeling

  • Evaluation

• Conclusion

+ Bonus: At the end of the article, you‘ll get a high-resolution cheat sheet with all the important aspects of ensemble methods.

So we’ve no time to waste. Let’s dive in!

Motivation behind it

The motivation behind ensemble methods:

• Reducing the variance: One of the primary goals of ensemble methods is to reduce the variance in the results. By aggregating the predictions of multiple models, the overall output becomes less dependent on the dataset.

• Reducing the bias: Another important motivation for ensemble methods is to reduce the bias in the model. When we combine different models, each with its unique strengths and weaknesses, the resulting ensemble can learn different classes more effectively than individual models.

🎯 Goal: Variance reduction without sacrificing bias!

Continue reading on Steady

Linear Regression is a simple statistical regression method, making it perfect for beginners in predictive analysis. It is a straightforward method, but some aspiring data scientists struggle to understand the mathematical aspects.

Terms like intercept or residuals occur? We want to bring light into the darkness!

There are many resources on the internet that try to explain this method. However, these explanations often use incorrect variable names or forget some indices on the variables.

Our goal is that you understand linear regression visually, mathematically, and practically. First, we explain the intuition of the linear regression visually. Then, we go deeper into the mathematical explanation. Finally, we look at a practical example from the economy. To get our data, we use the Python package OpenBB.

As a Data Scientist, it is essential to understand how this basic statistical method works. Always keep in mind, that you can often solve tasks with simple statistical methods before you need to use advanced ML models. For this reason, you should always try to solve problems with simple methods like linear regression before you use advanced ML methods.

We’ll discuss the following points:

• Visual Explanation

• Mathematical Explanation

• How can we calculate the Residuals?

• How can we calculate the Intercept and the Slope?

• Assumptions of a Linear Regression

• Common Evaluation Metrics

• Real-world example with economic data

  • Set up the environment

  • Data Understanding and Preparation

  • Modeling

  • Evaluation

• Conclusion

+ Bonus: At the end of the article, you‘ll get a high-resolution cheat sheet with all the important aspects of linear regression.

So we’ve no time to waste. Let’s dive in!

Visual Explanation

You can perform linear regression with multiple variables or just one. For the explanation, we’ll focus on using a single variable, so it’s easier to grasp. 😊

This form of linear regression is known as simple linear regression or univariate linear regression. In simple linear regression, we use one independent variable to predict a dependent variable.

The goal is to find the best-fit line that represents the trend in the data. For example, you can recognize trends in stocks or house prices.

In addition, you can use linear regression to understand the relationship between the two variables. It examines how the change in an independent variable influences the change in a dependent variable.

In the following, you can see the intuition of a simple linear regression:

Continue reading on Steady

In the financial sector, we often deal with time series data that have a DateTimeIndex on a specific time scale, like every day or every hour. Sometimes, it's necessary to aggregate this data based on frequency, such as monthly or quarterly. You might think that a simple groupby could solve this challenge. However, a groupby can only tackle part of the issue. For instance, it doesn't have the capability to understand business quarters.

This is where Pandas' time resampling feature becomes invaluable. In this tutorial, we delve into using this feature to analyze Tesla's stock market data. We'll showcase how to effectively manipulate time series data for insightful analysis.

We'll cover the following points:

• Technical requirements

• Load time series data with OpenBB

• Explore time series data

• Time Resampling

Continue reading on Steady

Time series are everywhere in the financial sector! When processing time series, there are so many points to consider. In this article, we dive right in and show you how to work with time series data using Pandas! We'll explore what a time series is and how to master the DatetimeIndex in Pandas. So, whether you're a data scientist or a beginner stepping into the thrilling world of data analysis, this article is packed with valuable insights you won't want to miss!

Continue reading on Steady

Machine learning (ML) has great potential for improving processes and products. A challenge is to explain the predictions of the ML algorithms. Trust and transparency are central arguments for the explainability of decision findings by an ML model.

In this article, we introduce the basic concepts of Explainable Artificial Intelligence (XAI). Furthermore, we present the properties or requirements of an explanation. In the context of this article, we use the terms interpretable and explainable synonymously.

Subscribe to newsletter

Continue reading on Steady

Do you need a secure server? In this article, you will learn how to set up an SFTP server with Docker. An SFTP server is a secure FTP server. You can store your data on an SFTP server, and the data transfer is encrypted. In this post, we will introduce the basics of SFTP and show how to set up a server with Docker.

The steps are the following:

1. Technical requirements

2. SFTP Basics

3. Set up an SFTP server with Docker

4. Conclusion

5. References and useful links

Continue reading on Steady

Do you know how to build an LLM application in a responsible way? No, then you should check out this article. We will cover some peculiarities we have learned in the last couple of months. You will take away some exciting facts from this article.

In our article, we will show you best practices for building complex applications using ChatGPT API. First, you will learn that LLMs like ChatGPT don’t see words. They work with tokens and this can lead to undesirable behavior during development. We will show a practical example of this. In addition, it is essential to check the user input for problematic content, such as hateful speech.

Further, your LLM system should also identify what type of query the user inputs. Such systems need multiple internal steps that are invisible to the end-user. In this way, you can teach your LLM to think in categories. The LLM decides which category was requested by the user. With this, you can significantly simplify internal processes. We will discuss an example that shows how it works.

We’ll cover the following points:

• Setup the environment

• How does Large Language models work?

• Think in categories

• OpenAI’s Moderation API

• Conclusion

Let’s start!

Continue reading on Steady

This article describes an example project in the context of stock trading. Can we predict with an AI model whether a stock goes up or down the next day? It is precisely the issue that we examine in this article. In this context, we compare the classification algorithms XGBoost, Random Forest and Logistic Classifier. In addition, the focus of the article is on data preparation. How can we transform the data so that the models can process it?

But before we start, we have to think about the problem statement and the financial data we need. In this article, we will follow the CRISP-DM process model so that we have a structured approach to solving the business case. CRISP-DM is particularly suitable for potential analyses and is often used in industry to structure data science projects.

As a data scientist, the first question you should ask yourself is, where do I get my data from? We will use the Python package openbb. The package bundles some data sources from the financial sector in a convenient way.

So let’s get started!

Continue reading on Steady

Project controlling is essential in data science projects, as the time required and costs are often difficult to estimate. Clients want to implement their use cases efficiently and successfully with machine learning methods. You can positively influence consistent project control with meaningful key performance indicators (KPIs). According to some studies, 85% of all data science projects fail, which requires early identification of obstacles.

In classical IT projects, many KPIs exist for project controlling, but these are not sufficient for data science projects. For this reason, in this article we present basic KPIs for IT projects and analyse them with regard to data science projects. As a result, we will see that specific KPIs can make the project controlling of data science projects more transparent.

The article first deals with the basics of project controlling and presents some KPIs from the software industry. Next, we use the presented KPIs and analyse them regarding data science projects using the process model CRISP-DM (CRoss-Industry Standard Process for Data Mining). The analysis also includes personal project experiences from data science projects.

Project controlling

Project control is a central component of project management and is used to support project decisions. It has the following objectives (cf. [1], p. 334):

• Coordination of project objectives

• Supporting project management and identifying deviations from the plan and their causes

• Evaluation of risks and initiation of measures to control or reduce risks.

The project controllers use instruments to control the project, including a cockpit. IT projects are usually managed on the basis of a project order. A project description usually defines the scope of the project and the expected project goals on the part of the client. However, the project description often only contains rough descriptions, which is why refinement is necessary.

Finally, we need to derive a roadmap with milestones and resource and cost planning. The milestones are essential in project controlling because they determine the timing of reviews and thus facilitate the classification of the work status in the overall project.

The KPIs to support project controlling are the main focus of this article. Experienced project managers usually rely on clearly defined and easily interpretable KPIs. In the next section we present a concept for deriving meaningful KPIs for IT projects.

Continue reading on Steady

Search is everywhere! We all use search engines like Google, Bing, or You.com every day. Moreover, we search for information in apps like YouTube or X. It’s hard to imagine our everyday lives without searching. Companies also use search systems to search their internal documents. Keyword Search is the most commonly used technique to build search systems. 

We’ll discuss the following points:

• How does the keyword search work?

• How do you use a keyword search system in Python?

• How can language models improve Search Systems?

We have no time to waste. Let’s get started.

How does the keyword search work?

First, we want to introduce the high-level concept behind keyword search. The following figure shows a simple example.

Example: Keyword Search (Inspired by [1])

Let’s say we have the query “What does the abbreviation EPS stand for in the financial industry?”. We are searching a very small archive that contains four sentences. We can compare how many words match between the query and the archive. The first sentence contains two words that are common to the query. The second sentence has the word “The”, which also contains the query. In the third sentence, no word is equal. The last sentence contains four words from the query. In the figure, you can see the number of words for each sentence on the right side. In addition, you can see that the fourth sentence has the most words in common with the query. A keyword search might retrieve that as the answer.

Continue reading on Steady

Do you know this? You want to learn something new, but you don’t know where to start. This is how we felt when we wanted to understand distributions mathematically. Yes, our professor explained all these distributions to us, but only mathematically, with formulas! Our professor didn’t explain it in an easy-to-understand way with visuals.

That’s why we wrote this article about the most important univariate discrete distributions. We want to explain you these distributions mathematically and visually. Our goal is that you understand the relationship between the maths and the distribution plot. In addition, we also present you an example for each distribution.

As a Data Scientist, it is important to understand how distributions work. Distribution assumptions are the backbone for some ML algorithms and are essential to solving statistical questions, for example in the insurance industry.

We’ll discuss the following distributions:

• Bernoulli Distribution

• Binomial Distribution

• Geometric Distribution

• Poisson Distribution

• Uniform Distribution

Let’s dive in!

Continue reading on Steady

Do you want to deploy your Plotly Dash App? Yeah, then this article is for you. You probably know the following situation. You create a great app and want to share it with your colleagues or clients. There are two options: You can show it to them locally or you can deploy your app. The second option is much more convenient. You only need to send a link to your colleagues and clients.

You can easily deploy a Dash App using Docker in a few steps. And the best is it’s free. No credit card is required! We show you how you can do it.

What are you waiting for? Let’s get started!

Continue reading on Steady

Everyone has been talking about Prompt Engineering! But it has several limitations! For example, the model always decides based on the training data (often a large amount of data from the Internet). Most models use general data. They are not specific to particular use cases. If you want to create large language models (LLMs) for specific use cases, you should think about fine-tuning.

In this article, we look at the LLM platform Lamini. Lamini is a platform for developers to build private and fine-tuned models. According to Lamini, you can fine-tune your LLM in three lines of code. Wow, that sounds incredible!

In our article, we’ll show you practical examples of when you should fine-tune your LLM. We also highlight the pros and cons of prompt engineering and fine-tuning. Be curious! You will learn many new things.

We’ll discuss the following points:

• Setup of the environment

• What is Fine-Tuning?

• Prompt Engineering vs. Fine-Tuning

• Fine-tuned LLM vs. Non-fine-tuned LLM

• Procedure for Fine-Tuning

• Fine-Tuning in practice with Lamini

• Conclusion

Continue reading on Steady

Since the release of ChatGPT, large language models (LLMs) are becoming more and more popular. New LLM-powered tools come on the market almost every day. You can also build some cool stuff with LLMs, such as Chatbots or document analysis. And the best, you don’t need to train LLMs from scratch. You can use pre-trained LLMs. 

LLMs have the potential to change many industries from the ground up. — Tinz Twins

Subscribe to newsletter

In the last months, many LLM frameworks for developers entered the market. LangChain is one of these new frameworks. Many developers use LangChain to build impressive applications. Take a look at the Github star history of the LangChain project.

Growth of LangChain, impressive! (Image by star-history.com)

Today, we introduce you to the basics of LangChain. This article is for those who have never used LangChain but want to learn it. We show you the basic concepts in a few minutes. Let’s dive in!

What is LangChain?

LangChain is a framework for building applications with the power of LLMs. The framework consists of components that are modular and easy to use. The components are abstractions for working with LLMs, whether you use the rest of the framework or not. Moreover, LangChain contains Off-the-shelf chains. You use these chains to combine several components. This approach makes it easy to develop complex applications.

LangChain has a lot of functionalities, and it’s hard to keep track as a beginner. The framework currently contains six main modules, and in this article, we want to explain the core features of each module. This approach helps you to understand the framework from the ground up in an easy way. 

Continue reading on Steady

On the Internet, you can often find PDFs or websites titled “The 100 Best Prompts to Boost Your Productivity.” These prompts are mostly for tools like ChatGPT to optimize simple repetitive tasks. Most of the time, you still need to customize these prompts to your individual use case, because each use case is different. That’s why you should learn to write effective prompts yourself, especially as a software developer. That’s exactly what this article covers.

Many software developers are not yet aware of the power of Large Language Models (LLMs) in software development. Prompt engineering, in combination with production-ready LLMs, will simplify software development massively in the future. LLMs make it possible for any developer to design AI-powered applications very quickly. It’s exciting to see how LLMs have become a kind of Swiss Army knife in the development of AI applications.

In this article, we’ll show you what you can do with LLMs and how you can control LLMs with prompting. In addition, you’ll learn the best practices for developing clear and specific prompts. 

We’ll discuss the following points:

• Initial Setup

• Types of LLMs

• Approaches for Good Prompts

• Conclusion

Continue reading on Steady

Don’t have enough time to watch a complete earnings call of a company? Maybe you have only a few questions. Then, it’s not necessary to listen to the whole earnings call. An AI chatbot can assist you by analyzing the YouTube video and answering your questions. This way, you can save time and still be well-informed.

In this tutorial, we implement an AI assistant with the framework LangChain, a very user-friendly framework for working with LLMs. And the best, you can follow the tutorial with basic Python knowledge. 
In addition, we’ll analyze the YouTube video of Tesla’s Q2 earnings call. But you can also use another YouTube video. Our approach works with all YouTube videos, you just need to enter the URL to your video.

Now, we’ve no time to waste. Let’s start with what you can expect in this tutorial.

What can you expect?

First, we’ll give you a technical overview. The overview guides you through the components that we need for our chatbot. In addition, we explain the loading of videos, the data preparation, and the retrieval process with LangChain.

Are you interested in what the finished application looks like? Here’s a demo of the final chatbot:

Impressive or? We’ve implemented this beautiful-looking web UI with Plotly Dash. The article focuses on the LangChain implementation and not on the web UI. Next, we jump into the setup process!

Continue reading on Steady

Do you often have problems with dependencies in your Python projects? Then, Poetry offers a solution for you. Poetry is a dependency management and packaging tool for Python. Dependency management is a fundamental aspect of Python projects. Good dependency management helps maintain projects and fix security issues.

Poetry addresses these points by managing the dependencies (install/update). It makes dependency problems a thing of the past, as it always finds a solution if one exists.

First, let’s look at why dependency management is so important. Then we install Poetry and look at the basics. In the last step, we integrate Poetry into an existing project. Let’s get started.

Why is dependency management so important?

Dependency management is crucial in software development. The following points list the advantages of dependency management:

• Code Stability: Managing dependencies ensures that the software works reliably and consistently. Effective dependency management avoids conflicts, incompatibilities and unexpected errors in the code.

• Collaboration with colleagues: Dependency management is critical when multiple developers work on the same project. It ensures that all team members use the same versions of dependencies, avoiding compatibility issues.

• Maintainability: Dependency management tools simplify the installation, update and removal of dependencies. This automation saves developers time, and the project is always up to date.

• Extensibility: Managing dependencies becomes increasingly difficult in large projects. With automated dependency management, you can integrate new libraries seamlessly into the existing code base.

• Continuous integration and deployment: With appropriate dependency management, you can realise automated pipelines for testing and deployment much faster.

These points show that appropriate dependency management is essential for software projects. Next, we look at the tool Poetry.

Continue reading on Steady

Do you often look for errors in software components that could prevent with a better testing strategy? Sometimes you or a colleague change a small thing in existing software, and a few weeks later, you realize that a certain functionality no longer works. It is frustrating and tedious to find the old bugs. You can avoid all this by testing the functions and classes of your code. When you make changes to the code, your tests immediately show whether the old functionality still works correctly.

Every developer makes mistakes, so you should test your code early. You should test new functionalities or changes to the code directly by writing a corresponding test. In Python, you can use the pytest package for testing. This article shows you how to write tests for functions and classes with pytest. We will also explain how to interpret the output of the pytest package so that you can quickly fix errors in your code.

We will discuss the following:

• Basics of pytest

• Testing of functions

• Testing of classes

Let’s start with the basics!

Continue reading on Steady

ChatGPT has impressively demonstrated the potential of AI chatbots. In the next few years, such AI chatbots will revolutionise many areas of the economy. Frameworks like LangChain make chatbot development accessible to everyone. But with these frameworks, you only develop the logic of the AI chatbot. The user interface is missing. A web interface is an elegant way to make a chatbot available for everyone.

But how do you create a custom web-based interface? Answering this question is the core of this tutorial.

Here’s a demo of the final GPT chatbot app.

Demo of the AI Chatbot (GIF by authors)

We’ll discuss the following topics:

• Why Plotly Dash?

• Technical requirements

• Project Structure

• Implementation of the AI Chatbot

• Conclusion

Why Plotly Dash?

Dash is written on top of Plotly.js, Flask and React.js. The open-source framework is licensed under the permissive MIT license. With Plotly Dash, you can build and deploy web apps with customised User Interface (UI) in pure Python. The framework abstracts the protocols and technologies needed to create a full-stack web app. This approach allows you to create data apps in a few minutes. From our point of view, Plotly Dash is the best choice to build web apps with Python. Do you like to learn more about the power of Dash and how to build Enterprise level web apps with Dash and Docker? Yes, then you can read our article about Enterprise-level Plotly Dash Apps.

Continue reading on Steady

Today, we’re talking about the best EDGAR library we have ever seen! We’ve tested many EDGAR libraries in the last year, and none of them have met our requirements. The libraries often have low functionality or have not worked properly. In addition, the libraries often had poor documentation. This article shows you a library that has convinced us!

Maybe you’re wondering what EDGAR even is. EDGAR stands for Electronic Data Gathering, Analysis, and Retrieval, and it is a database for legally required reports of all reportable companies. The United States Securities and Exchange Commission (SEC) manages this database. The filed reports are called SEC filings.

In this article, we look at the Python package edgartools, a very well-structured EDGAR library. We have tested the library, and it is very comprehensive. In addition, it is very powerful and easy to use. And the best, you don’t need an API key from the SEC. With this package, you can filter, query and select every filing since 1994. Furthermore, you can view the filing’s html, xml and text.

For instance, you can get Tesla’s current balance sheet with two lines of code:

Awesome! Now, let’s jump into the setup!

Continue reading on Steady

Do you want to compare many quarterly reports of public companies? It’s tedious to look through all the quarterly reports by hand. An assistant would be very helpful, wouldn’t it? That’s our motivation to build an AI assistant for comparing different earnings reports. Be curious!

We implement the assistant with the frameworks LangChain and Plotly Dash in Python. If you want to learn more about Plotly Dash, we recommend our article “A Comprehensive Guide to Building Enterprise-Level Plotly Dash Apps”. This article shows you how to build a production-ready web app with Plotly Dash and Docker. If you are a LangChain beginner, we recommend our introduction article about LangChain.

The article is a perfect starting point for understanding the basics of LangChain, and it’s essential for understanding the concepts in this article.

Now let’s start with what you can expect in this article.

What can you expect?

In this article, we’ll analyse the Q1–2023 earnings reports of Tesla, Mercedes Benz and BMW. First, we’ll give you a technical overview. The overview shows you the individual components which we need. Furthermore, we explain the loading of the PDF documents (earnings reports), the data preparation and how agents in LangChain work.

Do you want to know what the finished app looks like? Here’s a demo of the final web app:

We’ve no time to waste. Let’s jump into the setup!

Continue reading on Steady

Do you want to know how to create an anomaly detector using Python and TensorFlow? Then this article is for you. Credit card companies use anomaly detectors to detect fraudulent transactions. It is important to identify fraudulent transactions so that customers do not have to pay for something they did not buy.

Many credit card transactions take place every day, but very few transactions are fraudulent. The fraudulent transactions are anomalies. The article presents an implementation of an autoencoder model to detect these fraudulent transactions.

We’ll discuss the following points:

• Anomaly detection in general

  • Anomaly definition

  • Types of anomalies

  • Anomaly detection

• Autoencoder concept

• Credit Card Fraud Detection Implementation

• Data Preparation

• Modeling

• Evaluation

• Conclusion

Continue reading on Steady

Data Science projects always need some kind of visualisation. For the initial analysis, data scientists usually use Jupyter Notebooks and libraries like matplotlib or seaborn. In exploratory data analysis, data scientists use histograms, and scatter plots or make statistical evaluations. For the first insights, this approach is very well suited. However, interactive dashboards are more appropriate for presenting the results. Many clients want exactly that! Interactive dashboards are a proven approach to explaining results in an understandable and comprehensible way.

But creating an interactive dashboard is not a trivial task. In our view, Plotly Dash is the best choice for creating impressive diagrams. For a production-ready dashboard application, you must consider further aspects (e.g. deployment with Docker).

In this article, we want to share our best practices for building a well structured dashboard application with Plotly Dash. In addition, we show how to deploy a Dash App cleanly with Docker.

We’ll discuss the following points:

• Why Plotly Dash?

  • Advantages

  • Disadvantages

• Why Docker?

• Model View Controller Pattern

  • Advantages

  • Disadvantages

• Best Practices: Project Structure for Dash Apps

• Local Development

• Dockerize the Dash App (Production Environment)

• Conclusion

Continue reading on Steady

MLflow is an open source platform to manage the lifecycle of ML models end to end. It tracks the code, data and results for each ML experiment, which means you have a history of all experiments at any time. A dream for every Data Scientist. Moreover, MLflow is library-agnostic, which means you can use all ML libraries like TensorFlow, PyTorch or scikit-learn. All MLflow functions are available via a REST API, CLI, Python API, R API and Java API.

As a Data Scientist, you spend a lot of time optimizing ML models. The best models often depend on an optimal hyperparameter or feature selection, and it is challenging to find an optimal combination. Also, you have to remember all experiments, which is very time-consuming. MLflow is an efficient platform to address these challenges.

In this post, we briefly introduce the basics of MLflow and show how to set up an MLflow workspace on-premise. We set up the MLflow environment in a Docker stack so that we can run it on all systems. In this context, we have the services Postgres database, SFTP server, JupyterLab and MLflow tracking server UI. Let’s start.

Continue reading on Steady