Building with OpenSource Generative AI Training

• Train and optimize Transformer models with PyTorch.
• Master advanced prompt engineering techniques.
• Understand AI architecture, especially Transformers.
• Write and deploy a real-world AI web application.
• Describe tokenization and word embeddings.
• Install and use open-source frameworks like LLaMa-2.
• Apply strategies to maximize model performance.
• Explore model quantization and fine-tuning.
• Compare CPU vs. GPU hardware acceleration.
• Understand chat vs. instruct interaction modes

• Module 1: Learning Your Environment & Deep Learning Intro
  • Learning Your Environment
  • Using Vim, Tmux, and VScode Integration
  • Revision Control with GitHub
  • Deep Learning Intro
  • What is Intelligence? and Generative AI Unveiled
  • The Transformer Model Architecture
  • Feed Forward Neural Networks
  • Tokenization and Word Embeddings
  • Positional Encoding
• Module 2: Building and Training a Transformer Model
  • Build a Transformer Model from Scratch
  • Introduction to PyTorch
  • Construct and Orchestrate Tensors from a Dataset
  • Initialize PyTorch Generator Function
  • Train the Transformer Model
  • Apply Positional Encoding and Self-Attention
  • Attach the Feed Forward Neural Network and Build the Decoder Block
  • Transformer Model as Code
• Module 3: Prompt Engineering and Deployment Hardware
  • Prompt Engineering
  • Introduction to Prompt Engineering
  • Developing Basic, Intermediate, and Advanced Prompts (Chaining, Set Role)
  • Getting Started with Gemini (Hands-on exploration)
  • Hardware Requirements
  • GPUs role in AI performance (CPU vs GPU)
  • Current GPUs and cost vs value
  • Building with OpenSource Generative AI
  • Tensorcore vs older GPU architectures
• Module 4: Open-Source LLMs and Advanced Deployment
  • Pre-trained LLM & Deployment
  • A History of Neural Network Architectures
  • Introduction to the LLaMa.cpp Interface
  • Preparing A100 for Server Operations
  • Operate LLaMa2 Models with LLaMa.cpp
  • Selecting Quantization Level for performance and perplexity
  • LLaMa API Server & Applications
  • Deploy Llama API Server
  • Develop LLaMa Client Application
  • Write a Real-World AI Application using the Llama API
  • Constraining Output with Grammars
• Module 5: Optimization and Fine Tuning
  • Fine Tuning
  • Using PyTorch to fine tune models
  • Advanced Prompt Engineering Techniques
  • Testing and Pushing Limits
  • Maximizing Model Limits
  • Curriculum Path: GenerativeAI

Project Managers, Architects, Developers, and Data Acquisition Specialists. Ideal for Python Developers and DevSecOps Engineers looking to specialize in deploying and optimizing large language models (LLMs).

• Python - PCEP Certification or Equivalent
• Experience and Familiarity with Linux