SHORT COURSE DESCRIPTION

This two-day course is intended as both a theoretical and practical introduction to Automatic Item Generation (AIG), which is an emerging research area and an innovative assessment approach for generating assessment items using state-of-the-art technology. This course is designed for those who wish to learn the background, benefits, innovations, and practical applications of the item template and test development process of AIG. This course specifically integrates hands-on training on the AIG item template development to gain theoretical knowledge and practical experience on the process.

Day 1 of this course starts with laying the foundations of AIG, addressing the context, the basic structure of AIG, and the potential benefits and advantages of AIG. Next, the psychometric and practical implications of AIG will be covered, as well as challenges and potential solutions for the successful implementation of AIG.

Day 2 of this course first illustrates various assessment and measurement models that incorporate AIG, synthesizing the role and potential benefits of AIG within an Assessment Engineering framework and illustrating various assessment applications. The course then describes and explains the Computer Adaptive Formative Assessment (CAFA) web-based AIG tool, formulates innovations that CAFA AIG features, and points to potential benefits/advantages of CAFA AIG. A variety of CAFA AIG item template developments will be used to provide hands-on training to help understand/learn the item template development process. Participants will also be introduced to various Information and Communication Technology (ICT) applications, which synthesizes other technology-enhanced assessment components such as Automatic Scoring and/or Computer Adaptive Assessment into AIG. The short course will be concluded by highlighting future directions on AIG research and practice.

TARGET AUDIENCE

This course intended for anyone interested in researching and developing items, tests, and related-services using AIG techniques:

1. Students in graduate-level courses in psychological and/or educational measurement may find this course helpful for better understand several important phases in test and item development: AIG template development, AIG item delivery, AIG item validation, and test development with AIG items. Participants will be introduced to theoretical and psychometric implications of AIG.
2. Professionals those directly involved in developing test items and managing tests may find this course useful as source of expanding their present understanding of item and test development toward AIG, especially with technologically-enhanced and innovative item types. Participants will be exposed to practical and policy implications of AIG for assessment services.
3. Engineers those who are designing and developing technology-enhanced assessment-related services may find this course useful as source of identifying cost/benefits, strategies of sustainable development/managements of AIG services, and tips on developing assessment applications using AIG.

SHORT DESCRIPTION OF AIG

Modern researchers, psychometricians, item writers, and assessment service providers increasingly find themselves facing a new paradigm where the assessment item production process is no longer manual, but rather can be a massive production automatized by technology, that is, Automatic Item Generation (AIG). AIG is an emerging research area and an innovative assessment tool where cognitive and psychometric theories are integrated together into a comprehensive assessment development framework for the purpose of generating assessment items using state-of-the-art technology, especially in Information and Communication Technology (ICT) environments. The number of content areas and the number of applications of AIG are exploding. As such, this new reality raises important issues in effective item development.

REQUISITE KNOWLEDGE

It is assumed that participants have sound understanding of basic concepts of educational or psychological measurement, such as reliability, validity, test security, and the item development and/or item validation process. Although not required, a participant’s experience in this course will be enhanced by additional prior coursework or experience with other modeling techniques such as factor analysis, item response theory, structural equation modeling, and/or multilevel modeling.

DATES AND TIMES

February 23-24, 2017: 9:00-4:30 (check-in 8:30)

Continental Breakfast*:  8:30 AM – 9:00 AM
Morning Session:           9:00 AM – 12:00 PM
Lunch (on your own):    12:00 PM – 1:00 PM
Afternoon Session:        1:00 PM – 4:30 PM

* Participants who have special dietary needs or preferences are welcome to bring their own food as well.

LOCATION

Benjamin Banneker Room, room 2212
Adele H. Stamp Student Union
University of Maryland
College Park, MD  20742

Link to campus maps

COURSE FEES
(seats limited)

$495 fee for both days; $325 for full-time students (free for pre-registered HDQM Department faculty and students, although you must register).

ON-LINE OPTION: $175 for both days. See below for details.

ON-LINE OPTION

Format: On-line participants will receive a personalized login code to access a reliable live-stream of the workshop (with their personal computer), showing the instructor as well as the handouts displayed on screen to on-site participants.

Materials: On-line participants will receive electronic copies of the workshop materials that on-site participants receive, as well as any other relevant materials or information.

Timing/access: On-line participants may choose to watch the stream synchronously, or may elect to watch a recording of the workshop asynchronously, or both. With DVR-like capabilities, and access to the recordings for one-week after the end of the workshop, this format allows on-line participants to choose when they engage. This is especially useful for on-line participants in different time zones, or anywhere in the world, who may choose to watch at some later time than (but within a week of) the actual workshop time. (Asynchronous participation does not include real-time chat with other on-line participants, although a visual record of prior chats will be viewable).

Technical support: Participants are assumed to be able to secure a reliable computer, internet browser, and Wi-Fi connection. Challenges at the user end must be resolved by the user. Fortunately, because the workshop is recorded, users experiencing technical challenges can still “catch up” by watching the recordings to which they have access.

Content support: Content support for on-line participants is limited to real-time chat with the on-line participant community and any quantitative methodology doctoral students who might also be participating. There is no mechanism for on-line participants to submit individual questions to the instructor.

Hands-on activities: On-line participants may choose to try any hands-on activities being done by on-site participants using their own computational facilities and software; however, support for such activities is limited to the on-site participants.

Cost: $175 for two-days of streaming content, access to the workshop recordings for one week total, and electronic copies of all materials.

HOW TO REGISTER

Please register using the preferred on-line registration form:
On-line short-course registration form
(Note: For those students who are not in the HDQM department please fill out the paper registration form and upload it when prompted within the on-line registration system).

For those who prefer not to use the on-line registration system, please complete and submit the following paper registration form:
Paper short-course registration form

Note that it may take up to 7-10 business days for your registration to be processed.

QUESTIONS?

Contact Mr. Jordan Prendez -- umd.workshop@gmail.com

NEARBY AIRPORTS

DIRECTIONS: DRIVING

From Baltimore and Points North

• Take I-95 South to Washington, D.C.'s Capital Beltway (I-495).
• Take Exit 27 and then follow signs to Exit 25 (U.S. 1 South toward College Park).
• Proceed approximately two miles south on U.S. Route 1.
• Turn right into Campus Drive entrance (main gate).
• Proceed up Campus Dr to the "M" circle, go halfway around the circle and continue on Campus Dr (second right after entering the circle).
• Proceed up Campus Dr to the "M" circle, go halfway around the circle and continue on Campus Dr (second right after entering the circle) until you see Stamp Student Union on your right side.

From Virginia and Points South

• Take I-95 North to Washington, D.C.'s Capital Beltway (I-495).
• Continue North on I-95/I-495 toward Baltimore.
• Take Exit 25 (U.S. 1 South toward College Park).
• Proceed approximately two miles south on U.S. Route 1.
• Turn right into Campus Drive entrance (main gate).
• Proceed up Campus Dr to the "M" circle, go halfway around the circle and continue on Campus Dr (second right after entering the circle) until you see Stamp Student Union on your right side.

From Virginia and Points West

• Take I-66 East or I-270 South to Washington, D.C.'s Capital Beltway (I-495).
• Go East on I-495 toward Baltimore/Silver Spring.
• Take Exit 25 (U.S. 1 South toward College Park).
• Proceed approximately two miles south on U.S. Route 1.
• Turn right into Campus Drive entrance (main gate).
• Proceed up Campus Dr to the "M" circle, go halfway around the circle and continue on Campus Dr (second right after entering the circle) until you see Stamp Student Union on your right side.

From Annapolis and Points East

• Take U.S. 50 to Washington, D.C.'s Capital Beltway (I-495).
• Go North on I-95/I-495 toward Baltimore.
• Take Exit 25 (U.S. 1 South toward College Park).
• Proceed approximately two miles south on U.S. Route 1.
• Turn right into Campus Drive entrance (main gate).
• Proceed up Campus Dr to the "M" circle, go halfway around the circle and continue on Campus Dr (second right after entering the circle) until you see Stamp Student Union on your right side.

From Washington, D.C. (Northwest/Southwest)

• Take 16th St. North which becomes Georgia Ave. North at Maryland/D.C. line.
• Go East on I-495 toward Baltimore.
• Take Exit 25 (U.S. 1 South toward College Park).
• Proceed approximately two miles south on U.S. Route 1.
• Turn right into Campus Drive entrance (main gate).
• Proceed up Campus Dr to the "M" circle, go halfway around the circle and continue on Campus Dr (second right after entering the circle) until you see Stamp Student Union on your right side.

From Washington, D.C. (Northeast/Southeast)

• Take Rhode Island Ave. (U.S. 1 North) which becomes Baltimore Ave. North at Maryland/D.C. line.
• Proceed through the city of College Park.
• Turn left at Campus Drive entrance (main gate).
• Proceed up Campus Dr to the "M" circle, go halfway around the circle and continue on Campus Dr (second right after entering the circle) until you see Stamp Student Union on your right side.

VISITOR PARKING

Participants may park at the Union Lane Garage (located between the Adele H. Stamp Student Union and Cole Field House) for a daily fee. There are numerous metered spaces on campus but the University police are diligent about ticketing cars at expired meters as well as cars without appropriate stickers in reserved parking lots.

More information about parking on University of Maryland Campus can be found at the web site: http://www.cvs.umd.edu/visitors/parking.html

METRO (SUBWAY)

The Campus is conveniently located approximately 1 mile from the College Park-University of Maryland Metro Station. The stop is on the green line of the D.C. Metro System. The University of Maryland Shuttle Bus runs from the College Park Metro stop on a twenty-minute schedule through the Campus. Or, a brisk twenty minute walk up a moderate hill through the Campus will bring you to all locations.

D.C. Metro Map: http://www.wmata.com/rail/maps/map.cfm

ACCOMMODATIONS

Participants are responsible for arranging their own accommodations. For out-of-town guests, there are several sources of accommodations in the immediate area. Information about hotel pricing and reservations can be found at the web site: http://www.cvs.umd.edu/visitors/offcampus.html. Note that participants will need to make their own arrangements for transportation to and from campus.

[Note that there is a hotel located on the edge of the University of Maryland campus: The Marriott Inn & Conference Center, University of Maryland University College. For more information about this hotel, visit: http://www.marriott.com/hotels/travel/wasum-college-park-marriott-hotel-and-conference-center/

THE INSTRUCTOR

Jaehwa Choi is Associate Professor and Director of Assessment, Testing, and Measurement Program in the Department of Educational Leadership at the George Washington University. Dr. Choi holds a Ph.D. from the University of Maryland in the Department of Measurement, Statistics, and Evaluation in 2006.
His research interests and current projects include:

• Next Generation Assessment and Measurement Models: Truncated Logistic (a.k.a., PTL) Item Response Theory Models; Diagnostic and Formative Assessment Models; Learning and Assessment Models for Synthesizing Human and Artificial Intelligence
• Assessment Engineering Systems: Computer Adaptive Formative Assessment (CAFA); Automatic Item Generation; Automatic Scoring and Grading; Game-based Assessment;  Educational Big Data Analytic Systems; Pathway to Advancement and Success in Statistics (PASS) System
• Markov chain Monte Carlo Estimator (MCMC): Bayesian and MCMC estimation methods for various latent variable models, e.g., DL ANN, Structural Equation Modeling, Item Response Theory Models, or Hierarchical Linear Modeling

His research has appeared in such journals as Journal of Educational and Behavioral Statistics, Multivariate Behavioral Research, Structural Equation Modeling: A Multidisciplinary Journal, Applied Psychological Measurement, Journal of Modern Applied Statistical Methods, Journal of Counseling Psychology, Asia Pacific Education Review, and Journal of Rehabilitation.

He may be reached at jaechoi@gwu.edu