Assignments
Preparation for first class:
Read the MISQ article by Allen et al. (2006) and answer the following items:
What are technological methods that websites can use to block web agents?
What are legal methods that websites can use to block web agents?
Choose one website that you’d like to crawl. Go and find whether it has a robots.txt file, and if so, copy its content here.
What are the recommended steps that a researcher should take before deploying a web-crawler on a commercial website?
Assignment 1: Starting up your study (HW1 Study startup.docx)
In this assignment you will find instructions on populating your Wiki page, focusing on the research question.
Assignment 2: Web crawling (HW2 web crawling usefulness.docx)
You will find instructions on further populating your Wiki page, this time focusing on the value of the data that you plan to collect, given your research question.
I also suggest to go over the PHP crawling and parsing that we did in class and replicate it. The last class slide contains an exercise to try out on your own (no need to submit this).
Assignment 3: Web crawling, RSS, Web services (HW3 web crawling RSS API.docx)
Please submit three files:
The modified PHP files in the Web crawling part
The completed Word file for RSS and Regular Expressions
Your final PHP code for the API part
You will also find instructions on preliminary reading for next class (on surveys), and further populating your Wiki page, this time focusing on your population of interest.
Assignment 4: Sampling issues (HW4 Sampling from the web.docx)
In this assignment you will read about sampling issues that arise in Web crawling. You will also find instructions on further populating your Wiki page, this time focusing on sampling concepts.
Assignment 5: Questionnaire design and IRB (HW 5 Questionnaire Design.docx)
You will find instructions on proceeding with your project: filling the IRB form and creating your questionnaire in the online survey tool.
Assignment 6: Frame and pretesting (HW 6 Frame and Pretesting.docx)
In this assignment you will create the sampling frame for your survey, and pre-test your questionnaire.
Assignment 7: Solicitation email and survey pilot (HW 7 Solicitation Email and Piloting.docx)
This assignment focuses on your online survey: you'll find instructions on creating the solicitation email, on piloting your online survey, and on continuing to fill the IRB form.
Assignment 8: Sample design (HW 8 Sample size and estimation.docx)
You will find instructions on proceeding with your project: Choose your sample design and determine your sample size. Complete your IRB form. Then, draw the sample from the frame.
Assignment 9: Sample design: Stratifying (HW 9 Sample design.docx)
Assignment 10: Response Analysis (HW 10 Survey response analysis.docx)
Please update your Wiki according to the directions. We will examine your chart in class.
Assignment 11: DOE (HW 11 DOE ClockSpeed.docx)
Note: Please read the first 10 slides in the Advanced Designs deck (on random effects). Also, leave Q7 blank (unless you know the answer), and you should be able to fill it in during next class.
You will also find instructions on proceeding with your project:describe the manipulation in your study in DOE terms. Make sure to update your Wiki.
Assignment 12: DOE and data analysis (HW 12 DOE Brokerage and data analysis.docx)
In this final assignment you will find instructions on how to analyze and present the data. This final part is due next week, and make sure to also update your Wiki.