-
-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[committee] Potential survey of community members about lessons #2
Comments
This is an interesting idea! Would this survey be mostly targeting instructors or learners, in your opinion? |
Thank you @bsmith89 for the encouragement, |
Looking forward to discussing this more during our meeting. Are you (or anyone else) aware of existing data on:
|
Another thing that might be useful to add to a survey would be a question or two about the prioritization of lesson plan elements as well as possibly some workshop disaggregation to make it easier for instructors to customize the workshop content based on the needs of the learners and available time. |
Perhaps we can continue to use this thread and some meeting time to compile a list of what data we'd like to collect (and from whom). If we have a pretty specific request, I'm hoping that we can pass that off to the Carpentries core team to produce the survey and distribute it. @ErinBecker, @tobyhodges, is this something that there would be bandwidth for? |
if anyone in the group is good at webscraping, we may be able to provide a list of workshop website URLs - from which one could pull out the Schedule/Syllabus tables each time... |
Is this schedule scrape perhaps similar to the data that Erin said was now available? |
If I understood correctly (@ErinBecker please speak up if I got this wrong!), Erin was talking about data collected when people request a workshop and/or report a self-organised workshop. The form we provide includes a question about what kind of workshop is being taught (see. e.g. "Which Carpentries workshop are you teaching?" in the form for self-organised workshops). As Annajiat mentioned, some Instructors are nevertheless "mixing and matching" episodes from e.g. the two Python lessons, and that information would not be captured by these form submissions. The web-scraping approach would probably provide a more accurate and detailed overview of exactly what episodes were (planned to be) taught at each workshop. To be honest, the web-scraping approach probably adds only a small benefit on top of the form submission data, which I would expect to give you a fairly decent summary on its own of which lessons are being taught. But I mention it just for the small chance that someone on the committee thinks they could perform this kind of scraping without much effort! |
Going forward the new stats package would give us information on which workshop modules are being used most heavily, although probably not which modules are being combined together by instructors on an ad-hoc basis. |
I could see some value in unstructured answers also where instructors could quickly summarize their adjustments to the lessons. (As opposed to asking e.g. for detailed answers about each topic in each lesson.) Would definitely be biased in who takes the time to write it out, but I still think it would be interesting to know e.g. that >=20% of instructors skipped a particular shell episode or something. |
There was consensus during our Q4 meeting that this planned survey is worthwhile. @tobyhodges has also indicated that core staff have been discussing "developing a programme of assessment for Carpentries curriculum". Seems like now is a good time to get involved with that process. @annajiat , while you missed our Q4 meeting, you've indicated to me that you intend to stay involved in the CAC. Attendees at the meeting also wanted to know to what extent you would like to spearhead this survey development, as it was your idea initially. Who else would like to take charge of pushing this survey forward? (Either leading or co-leading depending on Annajiat's plans.) There was also general consensus during our Zoom meeting that this survey should, at its core, be designed to find out what lesson materials are actually being taught, and based on what constraints. Specifically, we want to know what materials are being cut out of workshops, and how this depends on workshop format. The goal of this survey will be to inform lesson development going forward, as well as standardized teaching practices (i.e. what do we tell new instructors about culling materials for time). |
I'm happy to assist @annajiat with the survey. |
It might also be interesting to ask instructors to indicate the most popular content (episodes, topics etc. with most questions asked?) within the lessons they teach, so future instructors will have a better idea of what learners find most useful for their research. |
Thank you, @bsmith89, for asking. We need to take a note that even after preparing the schedule, there would be some changes which may not be reflected back to the schedule on the workshop website. But that is some inaccuracy that we might be able to live with. At least, that may be more precise compared to the workshop request data. Coming to think of it, it might be interesting if we can compare these 3 data (workshop request, schedule on site, and instructor survey) to see how the plan evolves. However, that may be a separate project someday. |
We can build on @mxiang1 's idea to provide multiple popular templates for the schedule. Currently, alpha/beta lesson don't show up in the schedule by design. While it has its good sides, ensuring that the instructors are fully aware that the lesson is still in development. It also leads to some repeated extra efforts for modifying the schedule again and again. In another perspective, a popular lesson needs to be kept updated to remain relevant to an interdisciplinary audience while serving as an example of dissecting and improving design of remaining lessons. |
As a follow-up related to workshop format, |
Yes! This came up in our meeting discussion as well. The material that instructors cover is highly context-dependent and in particular workshop format matters. I also want to raise the issue of this survey becoming unmanageably large/complex. My inclination is to try to maximize the actionable information that we can get from as short/simple of a survey as possible. In particular, a shorter survey will help us to get a larger and more representative sample of instructors. @tobyhodges has mentioned information gathering initiatives by core staff, which might encompass our goals as well. I'm curious what the timeline for that is, and whether we should try to work through that mechanism instead of doing an independent survey. One option would be for the CAC to design an independent survey in the next 6 months but with a much reduced scope (e.g. what parts of the lessons are instructors cutting for time?) and then also contribute to a broader "programme of assessment for Carpentries curriculum" with the more involved questions. |
Hi everyone, in case we haven't met, I'm Kelly and I recently moved from Deputy Director of Instructor Training to Director of Assessment at The Carpentries. I'm very glad to see this conversation happening. I have also been speaking to @tobyhodges and @ErinBecker about how we can begin to understand what lessons (and parts of lessons) are being taught. This is a question that is relevant across our Lesson Programs and curricula, so I'm hoping we can take a coordinated approach to how we decide to answer this. As a first step, perhaps interested members of the CAC could meet with @tobyhodges and I to discuss what a survey (or other types of data collection) might look like. Let me know if you are interested in participating in such a meeting and I will send out a whenisgood. Thanks! |
Hi Kelly,
Count me in. I’d be happy to discuss more about the survey.
Best,
Menglan
… On Jan 5, 2023, at 10:31 AM, Kelly Barnes ***@***.***> wrote:
Hi everyone, in case we haven't met, I'm Kelly and I recently moved from Deputy Director of Instructor Training to Director of Assessment at The Carpentries. I'm very glad to see this conversation happening. I have also been speaking to @tobyhodges <https://github.com/tobyhodges> and @ErinBecker <https://github.com/ErinBecker> about how we can begin to understand what lessons (and parts of lessons) are being taught. This is a question that is relevant across our Lesson Programs and curricula, so I'm hoping we can take a coordinated approach to how we decide to answer this.
As a first step, perhaps interested members of the CAC could meet with @tobyhodges <https://github.com/tobyhodges> and I to discuss what a survey (or other types of data collection) might look like. Let me know if you are interested in participating in such a meeting and I will send out a whenisgood. Thanks!
—
Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJZY4I74QX4IPZHQ6AYSNH3WQ4HRTANCNFSM5QLLIEBQ>.
You are receiving this because you were mentioned.
|
I'd love to join, too! |
Hi all, thanks for your interest in a meeting. I have created this whenisgood poll so we can find a time that works for everyone. Tagging those who said they were interested, but anyone can feel free to join. @tobyhodges @bsmith89 @mxiang1 |
Wow, you are all fast! Thanks for filling out the poll! I have scheduled the meeting for Monday January 23 at 8 pm UTC (Find the time in your own timezone). If you filled out the poll, I sent you an invite using the email address on file with AMY. If you didn't get it or would like an invite, let me know! Looking forward to chatting with you all! |
Hi everyone, thanks for a great meeting today. For those who couldn't attend, I wanted to provide a little summary and a link to the meeting notes. If you are interested, please read the notes and share any feedback you have here. I'm excited to work on this with you all! We discussed that the objectives of this work will be to:
We talked about the best way to get this information and determined it is to take a few different approaches:
Next Steps
|
Should we conduct a survey for
The text was updated successfully, but these errors were encountered: