Archive for the ‘Computers in the Curriculum – OISE CTL 1606’ Category

[Please see digital academic workflow video at Screencast of a simple academic workflow. Scroll down just a wee bit to see the whole screen.]

The amount of lost time and lost productivity resulting from inadequate academic workflows affects not only individual researchers but their whole field. While there have not been many studies done on workflows, one survey (University of Minnesota Libraries, 2006) found that 36% of the social science faculty at the University of Minnesota used reference managers compared to only 8% in the humanities. If we can assume that even fewer faculty members are using efficient academic workflows, then the state of research is in need of an upgrade.

The same study (University of Minnesota Libraries, 2006) found that graduate students are 50% more likely than faculty to use reference managers. Thus the concept of an academic apprenticeship (McGowan, 2005) takes on a new meaning, with graduate students able to teach faculty a lesson.

Howison and Goodrun (2005, p. 3) write that the system for managing academic papers “is far from standardized and ranges from manual ‘fumbling’ through to quite sophisticated if users employ citation managers, such as Endnote or the variety of Bibtex management systems. Yet even the most sophisticated users experience difficulty in managing their files.” This, they write, is due not only to industry lagging behind in the area of integrated support for researchers, but due also to the incapacity of PDFs and most other documents to allow metadata to be included within the document itself. When we download a paper, we must download the metadata separately, whereas when we download music, the metadata is included in the music file. This kind of inefficiency, in addition to a lack of available commercial workflows for researchers, has led many of us to create our own workflows, with varied results.

This search for a simple yet effective digital academic workflow is not new. Hundreds of researchers (see e.g. Judson, 2008; Roberts, 2009) have written on the web about their attempts to set up and use a system that will:

1)    Find new articles of interest quickly and download them simply with their metadata

2)    Enable them to annotate, mark up, and take notes digitally, with no need for a paper copy of their documents

3)    Keep all their papers and other research documents in one place

4)    Provide a way to search through those documents in an efficient way

I currently use a very simple system of Google Scholar, BibDesk, Skim (all free) and Spotlight (which comes on all Macs) which is a very limited version of a much more powerful workflow called researchr (Haklev, n.p.) which was developed by Stian Haklev at OISE, University of Toronto. One day I hope to be running researchr (we have experienced some technical difficulties in running my version of researchr reliably (update October 2012 researchr is now working beautifully, thank you Stian!), but in the meantime my current very basic workflow does all of the above simply and efficiently. One advantage of my workflow is that it is so basic that I can run it by myself, with no need for computer expertise, and so can anyone else. It should be noted here that this workflow will only work on a Mac as BibDesk, Skim, and Spotlight are applications that work on Macs only, but similar workflows are possible on any computer OS.

Google Scholar

I chose Google Scholar to do my online searching for articles because it was faster and more efficient than using the University of Toronto’s (where I am a graduate student) online library (Sept. 2011 update: The U/T library system has been completely revamped and it now far faster and more efficient than it used to be). Google Scholar probably has the widest coverage of all online library search engines and provides information on the number of times an article has been cited, which most other libraries don’t (Hull, Pettifer & Kell, 2008). Google Scholar “aims to rank documents the way researchers do, weighing the full text of each document, where it was published, who it was written by, as well as how often and how recently it has been cited in other scholarly literature” (, n.p.). The efficiency of looking up a topic and finding the most relevant articles ranked top down is currently unmatched by other online library search engines. However, according to Jasco (2005), a big problem with Google Scholar is that millions of articles have not yet been indexed, so coverage is far from complete. Despite this, I find Google Scholar very useful and will likely use it for many years, along with the University of Toronto online library for backup.

In fact, with Google Scholar I can set my preferences (click on the cog-like spoked wheel to the right of “sign in” in the upper right hand corner of any Scholar page) for the University of Toronto’s Get It! service (any university student should be able to use their own library membership to do the same, just type in the name of your university and click search in the Library links section), which means that as soon as I am logged into the U/T library system, all articles that are available from the U/T library will be identified by a “Get it! U of Toronto” link. I just click on the link and I’m taken to the databases where the article is available. So the U/T online library is nicely synched to Google Scholar now, which is fantastic because the U/T library has access to many articles that are not available online.


BibDesk is a free and open-source reference manager that is under active development by a small group of volunteers (, n.d.). With BibDesk, your collection of PDFs and other documents is organized by author name, document title, date of publication, etc. so that finding documents is easy. BibDesk automatically imports keywords when it imports the citation data, but you can add your own keywords as well, which is something that I have started to do with painstaking commitment. By adding my own keywords, I can find information I am looking for much faster (please see attached video for a demonstration).

BibDesk can search titles but cannot search the PDFs themselves, which would be a very useful feature, but this is easily done on a Mac using Spotlight, Apple’s powerful search tool. BibDesk also allows automatic generation of references for Latex users, and there are programs (BibDeskToWord, BibFuse) that are supposed to allow for BibDesk to Word full reference generation as well.

I am relatively happy with BibDesk as finally all my articles are in one place and bedlam has been defeated (Judson, 2008). But I could get neither the BibDeskToWord program nor BibFuse to work. So at present my workflow cannot generate instant references like a program like EndNote (over $100 for the student edition) can. As well, Zotero (free) seems very interesting and useful, so I will check out Zotero and may possibly switch from BibDesk to Zotero in the near future. This workflow is an evolving workflow… which is as it should be. There will always be new and more efficient programs appearing on the market, and I’m keeping my options open.

Image 1. BibDesk Screenshot


Skim is a PDF reader that allows highlighting and annotation. The most useful Skim tool for me is the Note tool, which allows me to write my own notes within the document. In the screenshot below, I have highlighted text referring to a topic I am very interested in – mutual aid – and I have made mutual aid the title of the note. By tagging text in this way, when I do a search using Spotlight, this reference to mutual aid will turn up (even though the words “mutual aid” do not appear in the text) and thus I can find this and other important references to mutual aid quickly and easily. This feature of Skim has ended a problem I’d had for years with not being able to tag text and search those tags.

Image 2. Skim Screenshot

Spotlight is an incredibly powerful search tool that comes installed on all Macs and allows for very efficient searching of documents, webpages, and more. The handy combination of Google Scholar, BibDesk, Skim, and Spotlight has been a huge relief for me, as my workflow used to be terribly disorganized and now it is streamlined, smooth, and highly functional, if basic. I will be developing this workflow in the coming years and I hope that it will continually be getting more efficient and powerful. Please see my basic screencast (link at the top of this page) for a demonstration of this workflow in action.

In closing, I have not yet mentioned anything about Web 2.0 and its capacity for enabling collaboration between researchers. Blogs, wikis, social bookmarking, and other Web 2.0 tools are changing the way we interact, and making international collaboration possible. Once we get our own personal workflows in order, the time saved and the efficiency gained will lead to better and more productive personal research, and international networking with researchers in our fields is leading us to Research 2.0 – networked, collaborative research on a global scale.

The future is looking good for researchers! And the first step is finding a great academic workflow.


Haklev, S. (2011). researchr. Available at

Hull, D., Pettifer, S. & Kell, D. (2008). Defrosting the Digital Library: Bibliographic Tools for the Next Generation Web. PLoS Comput Biol 4(10): e1000204. doi:10.1371/journal.pcbi.1000204

Judson, O. (2008). Defeating bedlam. The New York Times [online version], December 16, 2008. Available at

Muldrow, J. & Yoder, S. (2009). Out of cite! How reference managers are taking research to the next level. PS: Political Science and Politics 42(1), pp. 167-172.

Roberts, B. (2009). Get started handling academic citations like a pro. Available at (2011). About Google Scholar. Available at

University of Minnesota Libraries (2006). A Multi-Dimensional Framework for Academic Support. University of Minnesota. Available at


In Drive (2009), Daniel H. Pink tells us that we are not motivated by external rewards (like grades, or money) but by a desire for autonomy, mastery, and purpose. Watch the RSA Animate video of his TED talk. It is mastery now that I want to focus on because I have just watched an exceptional TED talk by Salman Khan of Khan Academy.


Khan tells us that a major problem we have in school is that there is no goal of mastery. Students study a topic, a concept, let’s say basic division. They get lectures on it, read a few pages in a textbook, do some homework, do some exercises in class maybe, some more homework, and then a test. No matter how well or how poorly they do on the test, in the following class they will be starting a new topic, let’s say fractions. Now, if you didn’t do too well with division, you’re not likely going to do too well with fractions either. And this just snowballs with time until you get math-phobic kids who maybe wouldn’t have become math-phobic if they had just learned division (or any basic concept) thoroughly the first time.

Image from

Mastery is missing in school, and according to Pink, it’s one of the three things that most motivate all of us, including students. But how can we achieve mastery in a class of 30 or more students? There is a curriculum, there are tests, there is no time for mastery. But Khan says there is. He’s made short videos on 2400 topics from Addition 1 (2+5=?) to Green’s Theorem (is Green’s Theorem hard? I don’t know, I’m not there yet…).

Q: What’s so great about videos?

A: Videos provide time on task. Students can watch the video as many times as they need; rewind, rewatch, review, until they get it right. Videos provide the opportunity for mastery.

Khan suggests watching a video for homework. I’d add that you need to do some exercises right after the video to consolidate what you’ve just learned. If you can’t get the exercises right, watch the video again. Repeat as necessary. Get them all right.

In class, the next day, you do some exercises, and if you get them all right, maybe you can help someone else who is having problems with the same concept. The teacher, who doesn’t have to lecture anymore, can use 100% of her class time in 1-on-1 time with students who need help (compare this to the usual 5% Khan says is standard for most classes). Students can work at their own pace, at exactly their level. Khan found that when students are given time on their own to master a concept, they all eventually will, even if it takes a long time, and they later on catch up to the students who caught on faster. Imagine a class where you score 100% on every test before moving on to the next skill, topic, or concept. That’s mastery.

One of the most satisfying feelings I have ever experienced is reading something very difficult and very interesting. It takes a lot of time to get thru the reading, it’s a real commitment, and sometimes you have to re-read passages and take notes, it feels like your brain is working overtime. But when you get it, you really get it, and what a great feeling it is when you know that you have mastered a difficult concept. Those are my favorite articles and books, the ones that challenge me and force my brain to crunch thru hard topics.

I’ve signed up at Khan’s Academy (URL) and there are two reasons that I did this. One is for my own mastery. I’ve always loved math, and science, and I have been away from them for far too long. The other is my son, who, like me, loves math and science. At Khan’s Academy, I can be his coach, and check his progress, and help him when the going gets tough.

And me, I’ll be looking forward to some tough going of my own at a higher level. That’s when the challenge sets in, that’s when learning becomes its own reward.

I can’t wait to get started… 🙂


Pink, D.H. (2009). Drive. New York: Riverhead Books.

In an OISE grad course called Computers in the Curriculum (CTL1606) I was discussion leader last week and decided to use etherpads as a medium for a collaborative discussion to be followed by a collaborative writing session. Here is what happened and my thoughts on it.

1) Scheduling is a problem even when you use a tool like WhenIsGood. Of the 4 students in my group, 2 signed up right away and could have an etherpad meeting with no problems. 2 others signed up later and chose more limited times, one student chose one hour only, and it was that same evening! So there was no time for me to set it up. So that student ended up not participating in the exercise.

To set this up effectively, it would have to be done more in advance, and students could be requested to choose at least 3 times when they were free. And although WhenIsGood is a really great tool, just posting some times and having students write their names under the times would also work, and might work better in that when a student has posted their name, they feel some obligation to keep that time free, which may not be the case when they have posted many free times on WhenIsGood.

Point 1: Scheduling is difficult!

Solution: If I did it again, I’d ask students to organize themselves and choose times together.

2) Even when you type clear instructions, students may have their own ideas about how to do something. I wonder how much this has to do with the fact that it was grad students I was asking to participate in the exercise, and they are often in a position to lead, not follow, as many of them are teachers already.

For example, I posted the following instructions in the etherpad. “Please use the chat box on the lower left to discuss your ideas on these two questions. Please use this white pad to write the document you’ll post later as a note in our class forum. So based on what you discuss in the chat, you write a joint document here based on your chat that answers the questions.” But in both of the etherpad meetings, students didn’t discuss ideas first, they started typing on the whitepad right away. When I inquired about why they had skipped that part, it turned out that one of the students (who participated in both of the etherpad meetings that took place) had used etherpads before in a class but they had only used the whitepad part, not the chat. So that was how she handled this exercise too.

Point 2: Even when directions are clear, students may not follow them in the way you expect them too.

Solution: If there were a next time, I’d join the exercise! This way students would do it the way it was planned and maybe they’d realize the value of first discussing an idea and then writing about it.

3) There were some stability issues with all of the etherpads I used. We used PiratePad for collaboration between discussion leaders prior to Week 4, and our pad that we made is not longer accessible. Likewise of the 2 ietherpads used, one is available now for viewing and one is not. We used TitanPad only once but it’s still there! But one student said there were lag times in the texting. “[T]here were many times I had to delete a response because by the time I typed it into the message box, the topic had changed before I could press return.”

Point: Etherpads are not very stable. This is a big problem.

Solution: Use Google docs next time. (Now I am wondering if unstable etherpads are a conspiracy to force people to use Google docs! It was Google who released the code for etherpads, they must have known they were unstable!)

4. Even tho the students in my group didn’t use the chat, they still had good experiences and wrote collaborative texts. So the exercise was a success from this viewpoint. When I read their co-constructed text, it flows well, with the ideas of two people mixing together*. This is collaboration! So just when students don’t do things the way you were expecting, it’s the end results that matter most.

Point: Students don’t always do what you are expecting them to do.

Solution: Go with the flow! Give over some control of the exercise to the students. As long as the outcome is good, it doesn’t matter if it’s done in a different way from what you were expecting.

*Actually on one of the etherpads, one student posted and another student posted after her. The first student didn’t post onto the second student’s writing. This is not very collaborative. It may be the result of a misunderstanding on the part of the first student as to what collaboration is. But on the second etherpad, the two texts are interchanged, and both students are posting onto what each other had written.

5. Finally, collaboration occurs on many levels. In our discussion forum, we collaborate by posting our ideas and commenting on others’. In the etherpad exercises, students collaborated by co-constructing a written text together. In an online chat, ideas flow between people, and negotiation of meaning is done in realtime (like When ideas have sex, TED talk). My friend Stian would say that these are granularities of collaboration. And I would agree with that. Collaboration happens on many levels! I think that collaboration on all of these levels would be a good experience for anyone hoping to use collaborative exercises in their own classes.

Final Comment: Well, in taking risks, we learn! That was one of the ideas we discussed this week. I took a risk by using a tool that looked promising, but I didn’t know that stability would be such a problem, or scheduling. Now I know! And I’ll be chatting and using etherpads again. I may use a Google Doc plus chat as well to collaborate.

Update (August 2012): I no longer use etherpads. I use Google docs only because of their stability and Google docs’ many useful tools. More on Google docs later.

(Image from Collaboration Techniques) 1

Situating this post: I am on a flight from Hong Kong to Tokyo, listening to Astor Piazzolla.

The CSCL 2011 conference (see 2 below for a brief description of CSCL) was fantastic, with tech luminaries for keynote speakers (Erik Duval, Roy Pea, and Ed Chi, now a research scientist at Google) and an overwhelming array of presentations at a very high level of research that really boggled my mind.

One interesting part of this particular conference is that presenters were told to make a 5 or 10 minute presentation (depending on whether it was a short paper presentation or a long paper) and then we all broke up into smaller groups to talk with whichever presenter had caught our interest the most. This put a lot of pressure on the presenters as presentations delivered in the usual type of conference are usually 30 or 40 minutes long (in my experience), so presenters had to really compact their description and their findings into a tiny slot. But the collaboration afterwards was fantastic! In fact, I think it benefits the presenters themselves at least as much as the participants, as they got so much feedback (mostly positive, but some constructive negative feedback too) on their research.

In a typical conference, there might be a few questions after a presentation, but little actual dialogue, and I’ve often seen Q&A sessions where the question asked was not really answered by the presenter. In our collaborative discussions, topics of interest were discussed in depth, and I have to say that although I wish the presentations themselves could have been longer (they really were squashed 3), this represents a new and very participatory way for researchers to engage in dialogue. I learned at least as much from the collaborative after-sessions as I did from the presentations. So this heightened my awareness of the importance of collaboration in learning, which can be done in both online and f2f environments. My own personal experience in collaboration has been minimal. This needs to change (I’m working on ideas on how to change it).

This brings up the point that we as teachers need to be involved in collaboration before we can understand the value of collaboration and share it with our students. In a recent study by Austin, Smyth, Rickard, Quirk-Bolt & Metcalf (2010), researchers found that the level of engagement of teachers in collaborative learning had a positive effect on the success of collaboration of their students. This is very important! The less engaged in collaboration the teacher was, the less successful was the collaboration of her students. Therefore if we hope to reap the benefits of collaborative learning for our students, we had better get collaborating ourselves! An interesting parallel is with technology. It seems likely that the better our own personal experiences with technology are, the more likely we will be to use them in class successfully. So let’s get playing with a variety of technologies, and let’s get collaborating! Let’s collaborate on our learning experiences with technology! 🙂


Austin, R., Smyth, J., Rickard, A., Quirk-Bolt, N., & Metcalfe, N. (2010).  Collaborative digital learning in schools: teacher perception and effectiveness.  Technology, Pedagogy and Education, 19(3), 327-343.

1 I read recently that the use of images helps us to better internalize and retain information so I am going to be using images in posts whenever possible. In the physical version of Wired (July, 2011), I read, “When information is presented verbally, a person will retain about 10% of the message 3 days later; add a picture and retention soars to 65% (John Medina, cited by Carmine Gallo, p. 108). I wonder what the retention rate for written text is?

2 Computer-Supported Collaborative Learning, thus there was a focus on collaborative learning (CL) and how computers and technologies support and enhance CL.

3 All papers were given to us in pdf format on a flash drive on the first day of the conference, so even with the very short presentation time, we could get the whole paper that very evening if we wanted to (this was also a huge improvement over waiting for months for the proceedings to eventually be published.)