ENVIRON 790.30 Time Series Analysis for Energy Data
Spring 2023

Course Overview

Class Hours

T-Th 12:00 to 1:15 LSRC A247
To join class on Zoom click here.
Passcode: F2022

Instructor

Luana Medeiros Marangon Lima
Office: Gross Hall - 102K
E-mail: luana.marangon.lima@duke.edu
Office hours: Thursdays 10:00-11:00am (Gross Hall 102K or Luana’s Zoom), or by appointment.

Teaching Assistant

Yu Hai
email: yu.hai@duke.edu
Office hour: TBD

Communication

We will use Slack for communication. I will add all students to the slack workspace I’ve created for the class. Using slack will assure I never miss an email from you and will also keep us one text message away! You may use slack on your computer and/or phone.

Click here to join our slack workspace.

Course Description

Time series and forecasting methods continue to improve due to the enhancements in computing power and capability of dealing with larger data sets. This course will focus on time series analysis, modeling and forecasting, with emphasis on energy and environment applications.

Throughout the course we will use real data sets from the US Energy Information Administration (EIA), National Oceanic and Atmospheric Administration (NOAA) and the National Renewable Energy Laboratory (NREL). This course will use R for most statistical analysis. Lectures will feature R syntax and/or demonstrations using the R Studio user interface. Note that R and R Studio work on Windows, Linux, and Mac operating systems.

Energy Analytics usually involve getting data, parsing the data and transforming the data to a state where you can actually apply time series analysis. This work is better done in R, therefore the course will also cover a short introduction to Python.

Upon completion of the course, the students will be able to use R to carry out basic statistical modeling and analysis and fit a model to data. The goal of teh course is to enable students to learn from data in order to gain useful predictions and insights.

Course Format and Grading

The course consists of lectures at which we will discuss theory and applications. We will learn the time series concepts through data analysis projects. During the classes we will also dedicate some time to learn the statistical packages in R related to the topic as well as small group problem solving. Aside from the in class problems, there will be a set of assignments, a forecasting competition and a final project. Grades will be based on:

	Percentage
Assignments - A1 to A7	70%
A8 - Forecasting Competition	10%
Final Project	20%

The assignments involve applying concepts and tools learned in class to an specific data set or problem. Students might work together and help each other. However, the assignments are to be submitted individually. The table below shows possible due dates for the assignments.

Policy on late submissions: Assignments are due at 11:59pm. Assignments submitted at least 2 hours after the deadline will have 1 point out of 100 deduction by hour. After that, there will be a 5 points out of 100 deduction per day.

The final project could take several forms. If you have an interesting dataset, you may choose to work with it using existing methods and software tools to run your time series analysis. Another idea is to take some previously published data and analysis and use it as a starting point. You could simply take the data and do your own analysis. Or you may reproduce part of the published analysis, but in this case you will need to go further and try different models and analysis with the data. Make sure you clearly state the difference between what you have done and what was done previously. Students are encouraged to work in teams of two or three for a project.

There will be two short presentations of your final project. For the first you will present the data set you will use, what you plan to do with it and the project motivation. For the second presentation you will show the class the main results obtained throughout the analysis. Aside from the presentations, you are required to submit a final report as if you were writing a research paper. Describe the data sets, tools used and results. If the data set has been used before show what else you have done with it and compare with previous published results.

The final project grading will be weighted as follows:

	Percentage
Proposal Presentation	20%
Final Presentation	40%
Report	40%

Class Etiquette

You should take responsibility for your education. I expect students to attend every class and get to class on time. If you must enter the class late, please do so quietly. Retain from using phones and tablets for social media during class. Some classes will involve coding on your laptop. I expect you to focus on the assignment and refrain from any web browsing that may disrupt the progress of your work. Your classmates deserve your respect and support. We will likely have students from many different backgrounds and countries in this class and you should all feel comfortable and make each other comfortable while participating.

Nicholas School Honor Code

All activities of Nicholas School students, including those in this course, are governed by the Duke Community Standard, which states: “Duke University is a community dedicated to scholarship, leadership, and service and to the principles of honesty, fairness, respect, and accountability. Citizens of this community commit to reflect upon and uphold these principles in all academic and nonacademic endeavors, and to protect and promote a culture of integrity. To uphold the Duke Community Standard:

• I will not lie, cheat, or steal in my academic endeavors;
• I will conduct myself honorably in all my endeavors; and
• I will act if the Standard is compromised.”

Please add the following affirmation to the end of all assignments, and sign your name beside it: “I have adhered to the Duke Community Standard in completing this assignment.”

Land Acknowledgment

“What is now Durham was originally the territory of several Native nations, including Tutelo (TOO-tee-lo) and Saponi (suh-POE-nee) - speaking peoples. Many of their communities were displaced or killed through war, disease, and colonial expansion. Today, the Triangle is surrounded by contemporary Native nations, the descendants of Tutelo, Saponi, and other Indigenous peoples who survived early colonization. These nations include the Haliwa-Saponi (HALL-i-wa suh-POE-nee), Sappony (suh-POE-nee), and Occaneechi (oh-kuh-NEE-chee) Band of Saponi. North Carolina’s Research Triangle is also home to a thriving urban Native American community who represent Native nations from across the United States. Together, these Indigenous nations and communities contribute to North Carolina’s ranking as the state with the largest Native American population east of Oklahoma.”

Course Modules

The class topics are divided into twelve modules. There will be readings and/or recording associated with each module.

M1 - Getting started, Intro to TSA, R and RStudio
M2 - Autocovariance and autocorrelation
M3 - Trend and seasonality
M4 - Missing data and outliers
M5 - ARIMA Models
M6 - Seasonal ARIMA Models
M7 - Intro to forecasting
M8 - Model Performance
M9 - State space models
M10 - Advanced forecasting models
M11 - Model based scenario generation

Class Proposed Schedule

The proposed schedule below is subject to change. My initial plan is to cover all the material listed here but I might modify it if extra time is needed for some particular topics. I will update this table as needed during the semester.

Lecture	Module	Date	Topic	Homework
L1	M1	Jan 12	Introductions and Course Overview	Join Slack Workspace
L2	M1	Jan 17	Intro to Time Series Analysis, Intro to R and RStudio, Github, R Markdown	A01
L3	M2	Jan 19	Autocovariance and autocorrelation function
L4	M2	Jan 24	Partial autocorrelation function ACF and PACF in R
L5	M2 M3	Jan 26	ACF, PACF and plots in R Trend Component Estimation	A02
L6	M3	Jan 31	Seasonal Component - Stochastic vs Deterministic Trend
L7	M3	Feb 2	Trend and Seasonal component estimation in R	A03
L8	M3	Feb 7	Stationarity Tests: Mann Kendall, Spearman, Augmented Dickey Fuller	A04
L9	M4	Feb 9	Outlier types, detection, how to handle missing data	A05
L10	M4	Feb 14	A3 Solution Finish outliers in R
L11	M5	Feb 16	Intro to the Traditional Box & Jenkins Models - ARIMA family Stationary Models: AR and MA process
L12	M5	Feb 21	A4 Solution AR and MA order (poll) ARIMA(p,d,q) Models
L13	M5	Feb 23	ARIMA(p,d,q) Fitting ARIMA Models in R	A06
L14	M6	Feb 28	A5 Solution Seasonal ARIMA and Periodic ARMA Models	A07
L15	M6 / M7	Mar 2	Finish SARIMA in R Intro to Forecasting Averaging Techniques
L16	M7	Mar 7	Forecasting with ARIMA Models Forecasting in R
L17	M8	Mar 9 (remote assynchronous)	Watch recodings for M8 Work on Project, Team building	Project Proposal (2-3 slides)
-		Mar 14	Spring break no class
-		Mar 16	Spring break no class
L18	M8	Mar 21	Review Model Diagnostics Review Residual Analysis and Model Selection Model Performance in R	A08
L18	M9	Mar 23	Model Performance in R State-Space Models Bayesian Statistics	A08
L20	M9	Mar 28	State Space Models in R Go over Forecasting Competition Forecasting higher frequency time series	A09 - part I
L21	M10	Mar 30	Advanced Forecasting Models in R	A09 - part II
L22	M11	Apr 4	TBATS models in R Scenario Generation	Work on project/competition
L23	M11	Apr 6	Scenario Generation in R Course Recap Course Evaluation	Work on project/competition
L24	-	Apr 11	Final Project Presentations Presentation Schedule	Work on project
-	-	Apr 13	MEM Symposium - no class	Work on project
L25	-	Apr 18	Final Project Presentations Presentation Schedule	Submit project and competition knitted files

Assigments Schedule

You should use Sakai to submit your work. Assignments should be submitted using the Assignments tab. This is a tentative schedule.

Assigment	Due Date
A01 - Course Set Up Sakai link GitHub link	Jan 24
A02 - ACF & PACF Sakai link GitHub link	Feb 3
A03 - Trend and Seasonality Sakai link GitHub link	Feb 10
A04 - Stationarity Tests Sakai link GitHub link	Feb 20
A05 - Decomposition Sakai link GitHub link	Feb 27
A06 - AR and MA models Sakai link GitHub link	Mar 6
A07 - ARIMA models Sakai link GitHub link	Mar 20
A08 - Model performance Sakai link GitHub link	Mar 27
A09 - Forecasting competition Sakai link GitHub link Kaggle link	Enter competition and create Github repo by Apr3rd (Part 1) First model by Apr 7th (Part 2) Final submission + knitted file by Apr 28 (Part 3)
Final Project Presentation & Short report	Apr 28