### COURSE DETAILS

The financial industry has been adopting AI and machine learning at a rapid pace. Alternative datasets including text analytics, cloud computing, algorithmic trading are game changers for many firms exploring novel modeling methods to augment their traditional investment and decision workflows. As more and more open-source technologies penetrate enterprises, quants and data scientists have a plethora of choices for building, testing and scaling models. While there is significant enthusiasm, model risk professionals and risk managers are concerned about the onslaught of new technologies, programming languages, and data sets that are entering the enterprise. With little formal guidance from regulators on how to validate models and quantify model risk, organizations are developing their own home-cooked methods to address model risk management challenges.

In this course, we aim to bring clarity on some of the model risk management and validation challenges with data science and machine learning models in the enterprise. We will discuss key drivers of model risk in today’s environment and how the scope of model risk management is changing. We will introduce key concepts and discuss aspects to be considered when developing a model risk management framework incorporating data science techniques and machine learning methodologies in a pragmatic way.

### Learning Objectives:
Upon completion of this course, you will be able to:

- Role of Machine Learning and AI in financial services
- Model Risk Management challenges and best practices for machine learning models
- Validating machine learning models: Quantifying risk, best practices and templates
- Regulatory guidance and the future
- Practical case studies with sample code

### Delivery:

- **LIVE**: Email [info@qusandbox.com](mailto:info@qusandbox.com) for upcoming LIVE training dates
- **ON DEMAND**: Pre-recorded sessions with interactive videos, slides, demos and fully functional code through Qu.Academy.

### Who should attend?

- Model Risk professionals, Model validators, Regulators and Financial professionals new to data-driven methodologies
- Quantitative analysts, investment professionals, Machine learning enthusiasts interested in understanding model risk and governance aspects in fintech, insurance and financial organizations

### Guided Exercise

Participants will go through a guided exercise to perform model validation on a chosen machine learning model of their choice. Guidance will be provided in scoping and implementing the project.

### Delivery

LIVE/ON DEMAND

### Number of Modules

7 modules

### Each Module

1.5 hours/module

### Registration options

- **ON DEMAND**: Access now through Qu.Academy
  
  [**Register**](https://buy.stripe.com/3cscQd0lb4aG50ceV0)

### MODULES

#### MODULE 1: Machine Learning and AI: A Model Risk Perspective
- Drivers of Model Risk in the age of data science and AI
- Machine Learning vs Traditional quant models
- How has the world changed?
- A tour of Machine Learning and AI methods
- Supervised vs Unsupervised Learning (Regression, Neural Networks, XGBoost, PCA, Clustering)
- Deep Learning & Reinforcement Learning (Keras, Tensorflow, PyTorch)
- Automatic Machine Learning & Machine Learning APIs (Google, Comprehend, Watson)
- ML on the cloud vs On-prem
- Models redefined: Data, Modeling environment, Modeling tools, Modeling process

#### MODULE 2: Model Risk Management for Machine Learning Models - Part 1
- ML Life cycle management
- Tracking
- Metadata management
- Scaling
- Reproducibility
- Interpretability
- Testing
- Measurement
- Performance Metrics and Evaluation criteria
- Model Inventory and tracking

#### MODULE 3: Model Risk Management for Machine Learning Models - Part 2
- Integrating Data Governance and Model Governance
- Development Models vs Production Models
- Fairness, Reproducibility, Auditability, Explainability, Interpretability & Bias
  - How do we objectively measure these?
  - Review of the Apple-Goldman Sachs credit card debacle
- Machine Learning options and considerations
- ML and Governance: Roles and Responsibilities redefined

#### MODULE 4: Pragmatic Model Risk Management for AI/ML models
- Challenges and best practices for pragmatic model management within the enterprise
- Working with open source projects
- Working with vendor models and machine learning APIs
- Quantifying model risk for machine learning models
- Model risk management for deep-learning models
- Validation criteria and best practices
- Templates for Model Validation for machine learning models

#### MODULE 5: Hands-on Case study
- Validating a Credit-risk machine learning model
- Working with Regression, Neural Networks, and Random Forest models
- Sample templates and worksheets will be provided

#### MODULE 6: Guided Exercise, Part 1: Scoping and design
- Participants will go through a guided exercise to perform model validation on a selected ML model.

#### MODULE 7: Guided Exercise, part 2: Demonstrate your skill
- Participants will demonstrate their findings to the class and obtain feedback from instructors and industry participants.

### PAST ATTENDEES
Past Attendees of QuantUniversity workshops include Assette, Baruch College, Bentley College, Bloomberg, BNY Mellon, Boston University, Datacamp, Fidelity, Ford, Goldman Sachs, IBM, J.P. Morgan Chase, MathWorks, Matrix IFS, MIT Lincoln Labs, Morgan Stanley, Nataxis Global, Northeastern University, NYU, Pan Agora, Philips Health, Stevens Institute, T.D. Securities and many more.

### PARTICIPANT FEEDBACK
- Sri was an awesome instructor. He took care of the group's needs and offered different channels to pose questions.
- Very competent and passionate about this very vast subject and caring about the learning of his students.
- I really loved the format of this course. The flexibility with respect to time was great as the modules were prerecorded and the possibility to pose questions to the instructor in scheduled Q/A meetings was extremely useful.

### KALIBA BILALA
#### CFA
- I truly enjoyed your recent webinars with the CFA Institute and also learnt a lot.

### Frederic Siboulet
#### Managing Director  Ernst & Young Global Consulting Services
- I loved the course. Very well organized, clear and concise delivery, with a lot of practical examples.

### Michelle Allade
#### Director, Model Risk Management MetaBank
- Comprehensive nature of the curriculum and the layout/sequence, which flowed very naturally. Sri did an effective job of introducing the big picture and alternating between focus on the building blocks and relating those back to the bigger picture.