Data Analytics, Big Data and Blockchain 25418

Data Analytics, Big Data and Blockchain (DABDB) course (code 25418)

Lecturer: Dr. Paolo Coletti Paolo.Colettiunibz.it . Office E203 - Office hours: www.paolocoletti.it/timetable
Website: www.paolocoletti.it/bigdata

If you are interested in Advanced Data Analysis optional course (6 credits, 36 hours, first semester)

If you are lazy, do not read below and just watch this video here

Starting from this academic year this course has become a free choice course for students of the Master in Entrepreneurship and Innovation as well as for the studentes of the Master in Accounting and Finance. Therefore, to better accommodate your needs, I will adjust the timetable depending on the students who plan to attend. So, if you choose this course and plan to attend it, either come at the first lesson or drop me an email.

The course in brief

This course gives you the skills to organize and analyse data in a rigorous and scientific way. This is a kind of skill which you might have seen in other courses, but probably either too theoretically or as part of other scopes. In this course you will learn the vital skill of taking raw data and extract useful information from them. All the course is hands-on, i.e. we will focus on examples, exercises and assignments rather than on theory. At the end of the course you will be able to gather unstructured data, put them into a usable format, analyse them finding relations and interesting features and present your results.
If you plan to study regularly during the course, you can skip 94% of the exam study load. For the first part you are evaluated on a group assignment without the need for mandatory attendance, while for the second part you are evaluated on regular courseworks and oral presentations of your home activities (mandatory attendance either in person or via Skype is required).
If you cannot study regularly during this course, videos of all my lessons are provided on YouTube. However, take into consideration to do the assignment for first part anyway, since it does not require attendance and you can skip 46% of the exam study load.

Exam information

We start with the throretical questions on cryptocurrencies and then, after sending away all the students who have finished, we will go on with the practical parts.

Before the Python and smart contracts exam:

You will not receive extra time to fix your computer issues during the exam (which should have been fixed weeks before if you studied for this exam), so come with a ready computer:

if you do not use your own notebook, borrow a computer from the library well before the exam and configure it before the exam
make sure that Anaconda is installed and that you can use Jupyter for Python;
make sure Metamask is installed, that you have some Rinkeby ethers on its accounts and that it interacts correctly with Remix;
check that you are able to locate \\ubz01fst.unibz.it\Courses directory;
just in case your notebook crashes the night before, some days before the exam borrow a computer from the library and check that you are able to log in on your unibz account and that you have, or are able to install, everything you need

In case of problems, feel free to write an email to me as soon as possible. Writing to me the night before the exam, even though it might happen that you stumble upon a problem at the last minute, does not give me the impression that you started to study well in advance...

What will be asked on cryptocurrencies

Generic lesson: I will extract two random topics from the video and make two open questions on them, to answer with pen/pencil on paper. Closed books.

IOTA technology: I will ask a use-case, your choice. I expect a half page about it, focusing on how that company uses IOTA technology and not on the history or non-IOTA related business of the company. Second question will be on how does the tangle work. If you want further information, you can get them from https://www.iota.org/research/meet-the-tangle . Both questions are to be answered with pen/pencil on paper. Closed books.

Course

Prerequisites

In order to correctly follow this course each student is required previous knowledge on these topics:

English understanding and reading: you must be able to read and understand spoken English at level B2. If, for example, you have problems reading this website, you will not follow the course correctly.
Microsoft Windows or Mac: you must already know how to use an operating system in its basic aspects
File handling: you must have a very good practice on how to move, delete, rename, copy, compress, uncompress files and directories. These are activities with which some students have severe problems.
Internet: you must be able to navigate, send and read email, use basic UNIBZ network features and download files.

Course content

precourse: things that you must already know (how to use the operating system, files handling, unibz network features) and will thus not be explained during the lessons, but it is better to take a look at the videos
basic Python
financial applications with Python
cryptocurrencies and blockchain technology (2 hours, 20th March)
IOTA by guest speaker Antonio Nardella (2 hours, 10th April)
smart contract on Ethereum blockchain

How to study for this course

This course is different from the majority of courses you are used to. This course is much more technical than theoretical and it is strictly sequential. This means that you have to adapt your study strategy. First of all, you either attend all the lessons (or compensate for missing lessons watching immediately the corresponding videos or reading the book) or it is really not worth coming to the next one, since you will have a hard life understanding the next topic. Moreover, after each lesson you must repeat slowly on your own everything done in class in order to be sure to have fully grasped the explained concepts before the next lesson. And, needless to say it, do the exercise without copying them from your colleagues.

For the exam the main difference with respect to other courses is that you have to train much more than studying. The content of this course is easy and does not need extensive study, however it is only with practice that you become skilled enough and know immediately what to do without wasting time.

Exam

Exam is split in three parts. The grade is the weighted average of the three parts, based on how many lessons were dedicated to each of them (in A.Y. 2018/19 it is 2/18 on cryptocurrencies, 13/18 on Python and 3/18 on smart contracts).

1. Theoretical questions on cryptocurrencies, blockchain technology and IOTA.
2. Practical exercises on Python and financial applications
3. Practical exercises on smart contracts on ethereum blockchain

Coursework

Attending students have the opportunity to skip part 2 of the exam and replace it with a constant coursework. The coursework consists in
(a) home exercises which will be assigned after each lesson and which must be returned to dr. Coletti via email not later than 24 hours before the next lesson (0 hours before the next lesson in case it is the day after);
(b) oral presentation of exercises and similar tasks at the beginning of each lesson. Attendance is mandatory to all Python lessons except one, as you must be available for being called. In case of problems, you can be available via Skype, but set up everything in advance as all the technical problems are your responsability;
(c) practical mid-terms which will be organized periodically during the course. In case you skip one, we can try to arrange to do it later, but I do not guarantee.
In order for the coursework to be considered sufficient, you must have fulfilled all three points and with a weighted average grade of at least 60%. If the grade you receive is sufficient but too low for you, please write an email to dr. Coletti at least 7 days before the exam telling him that you will do the full exam.
Note that the only required attendance is when you might be called for the oral presentations or at the mid-terms, for the rest of the lesson you stay only if you think it may be useful for you, there are videos covering all my lessons.

The same procedure applies, independently, for part 3.

Study resources

Topic	Lessons' slides	Videos as replacement of attendance	Written material as support
Precourse	No slides anymore in 2019	Go down here	No books in 2019!
Python	No slides anymore in 2019	Go down here	Overview of lessons topics
financial applications with Python	No slides anymore in 2019	Go down here	No books in 2019!
Cryptocurrences	Slides in PDF and in PPTX	Go down here	No books in 2019!
IOTA	Slides in PDF	Go down here	No books in 2019!
Smart contracts	No slides anymore in 2019	Go down here	No books in 2019!

Files and programs used in class	Last updated
All the files in a single package

Python 3 versus Python 2

Issue	Python 2	Python 3
print	it is a command: print string	it is a function, therefore it needs parentheses: print(string)
print with trailing comma	a trailing comma after print suppresses new line: print string,	to suppress new line use: print(string, end="")
integer division	5/2 gives 2. To get the float result: either 5.0/2 or float(5)/2 or 5/2.0 or 5/float(2); float(5/2) still returns 2.0	5/2 gives 2.5
range function	range function returns a list	range function returns a range object. To get a list: list(range(arguments))
xrange function	xrange function returns a range object, like range in Python 3

Videos of lessons

course advertisement		Old course advertisement
course brief presentation		Brief course presentation
precourse 01		Precourse for Windows 10 on unibz network and file handling, first part.
precourse 02		Precourse for Windows 10 on unibz network and file handling, second part.
precourse Mac 01		Precourse for Mac on unibz network and file handling, first part.
precourse Mac 02		Precourse for Mac on unibz network and file handling, second part.
01 installing Anaconda		For Python we will use Anaconda with Jupyter and Python 2.7 (but 3.7 is fine)
01bis refusing to install Anaconda		For those who do not have a notebook (and borrow one from the library) or refuse to install Anaconda
02 variables
03 operators
04 conditionals
05 functions
06 lists
07 tuples and dictionaries
08 loops
09 counting and summing
09 bis extra exercises on counting and summing
10 modules and packages
11 arrays and random
12 importing data		Do not forget to download the data for this and the following videos.
13 rate of return
14 portfolios
15 risk
16 an example of big data
17 Markowitz theory
18 CAPM and Sharpe ratio
19 simple regression
20 multivariate regression
21 Monte Carlo for predicting gross profit		PAY ATTENTION THAT COGS MUST have cogs=rev*numpy.random.normal(0.6,0.1,iterations), which is the opposite of what I say. Sorry.
22 Monte Carlo for predicting stock prices
Cryptocurrencies and blockchain technology		A decentralised currency, basic cryptography, Bitcoin history and technology, blockchain technology, advantages and criticisms
Distributed Ledger Technology IOTA Technology, Governance and Use Cases		by Antonio Nardella.
01 smart contracts introduction
02 Metamask and Remix		You need to install Chrome or Firefox. Safari and Edge do not work :-(
03 first Solidity contracts
04 a decentralized unstoppable bank
05 purchase contract
06 cryptotoken		THIS LESSON IS NOT ASKED AT THE EXAM (even though it is the most interesting smart contract...)
networkfolders.avi 24 MB		This short video illustrates how to reach unibz network folder \\ubz01fst (which contains course_coletti and your own personal stuff) using VPN when you are connected from outside university or when you are connected using wifi. This procedure is not part of exam's stuff.

Exam

Before the exam:

if you do not use your own notebook, borrow a computer from the library well before the exam and configure it before the exam
make sure that Anaconda is installed and that you can use Jupyter for Python;
make sure Metamask is installed, that you have some Rinkeby ethers on its accounts and that it interacts correctly with Remix;
check that you are able to locate \\ubz01fst.unibz.it\Courses directory;
just in case your notebook crashes the night before, some days before the exam borrow a computer from the library and check that you are able to log in on your unibz account and that you have, or are able to install, everything you need

Frequently Asked Questions

Q: Which software do I need for Python on my notebook?
A: Anaconda is available for Windows and for Mac. Take care to install Python 2. Watch the video 01 or 01bis for details.

Q: Which software do I need for smart contracts on my notebook?
A: We will use only Metamask, which is a browser add-on. Unfortunately only Chrome, Firefox and Opera accept add-on, so you need to have one of these three browsers. Watch video 02 on smart contracts for details

Q: I have no notebook. What do I need for the course and for the exam?
A: You can borrow a notebook from the library. Watch the video 01bis to know how can you use Jupyter, you have several possibilities. For smart contracts, probably before every lesson you will need to install the add-on to the browser with your 12-words phrase.

Q: How can I reach network folder \\ubz01fst from outside unibz or connected via wifi?
A: For Windows users: if you are connected to wifi ScientificNetwork try to digit in any explorer address bar \\ubz01fst.unibz.it and see whether you reach it. You need to provide your login and password, but you need to tell to your computer that you are using a different domain and then you have to type unibz\loginname instead of simply loginname. If this fails or if you are no connected to ScientificNetwork, then you need to install VPN. There is a specific video up here.
For Mac users: if you are connected to wifi ScientificNetwork, Finder -> Go -> Connect to server -> smb://ubz01fst.unibz.it . You need to provide your login and password, but you need to tell to your computer that you are using a different domain and then you have to type unibz\loginname instead of simply loginname.

Q: May I fix an appointment to talk with you?
A: I live in the XXI century, the age of asynchronous communication. Try to send an email to me clearly stating the course you are talking about (I have several ones) and the problem you have. You will be surprised by how fast I react, and without having the notifications on my smartphone.

Q: When will the next exam be? Can you give me a hint on the exam's date because I have to catch a plane? Can you move the exam's date? Can you fix the exam's date on the week I suggest?
A: Please stop writing me emails on this topic. Exam's date appears on your timetable as soon as it is official. If you have something to say about it, contact your students' speaker who is the only one who can submit requests on students' behalf.

Q: I may not enrol online for technical or administrative reasons or I forgot to enrol or it is my third attempt and I cannot enrol. Can I do the exam anyway?
A: No, I may not let non-enrolled students take part of the exam. Do not ask me to do illegal things! Ask the secretary whether there is something they can do.

Q: May I do the exam with my computer?
A: Sure. But beware: (1) you must be able to navigate the Internet and to enter directory \\ubz01fst.unibz.it\Courses\Course_Coletti. Do not wait for the day before the exam to check it. (2) You are responsible for your different programs' versions and configurations and for the absence on your computer of specific programs.
In any case you will have a unibz desktop computer in front of you.

Q: May I use the operating system in a different language?
A: Yes, sure. Your business.

Q: Will the exam be similar to the other exams on this website?
A: Sure. For the first year (2018/19) I shall put a couple of exam's prototypes.

Q: I lost a file during the practical exam because I did not save it correctly. What may you do?
A: Absolutely nothing. With time spent on exercises you should know the unreliability level of your programs, and how often you have to save.

Q: My files were not copied correctly at the end of the practical exam. What may I do?
A: Checking that the copy is correct, and practicing file copy even during the exam, is your task and is official prerequisite for this course.

Q: Hey, exam's time is not enough! I could not even finish it. If I only had other 5 minutes I would have done it much better!
A: Sorry but you are wrong, as I calculate more than twice the needed time. Look at the important warning after exam's explanation: the fact that for you exam's time was not enough is instead an indication that you must do many more exercises to be efficient and fast enough. On the other hand, if you have documented medical problems that slow your operations, write an email to me to have more time.

Previous exams

ID	Session	Exam link	Solution link
03	May 2019	exam	solution
02	prototype for A.Y. 2018/19	exam	solution
01	prototype for A.Y. 2018/19	exam	solution

This page is maintained by Paolo Coletti.

Marisa Crucitti il teatro per ringiovanire Paolo Coletti personal page La stanza dell'arte Paolo Coletti Paolo Associazione culturale e ricreativa Kender Trento Bolzano La stanza dell'arte Marzia Centro Felix Trento Aarghen Thael Il Vecchio Continente GURPS Marisa Crucitti il teatro per ringiovanire Laboratorio d'arte Gabbana cornici Rovereto Nursing Up sindacato infermieri Bolzano ASL Italia Advanced Squad Leader Club scherma Bolzano Bozen Fecht club spada fioretto sciabola