Public Lab Research note


Low floor, high ceiling for Public Lab research notes

by warren | July 01, 2015 15:41 01 Jul 15:41 | #12027 | #12027

Since the very beginning of Public Lab, we've been aware that we're experimenting with new modes of production, and that the means of publication and communication we employ are keys to success. We started with a range of different "types" of content on the first Public Lab site, including:

  • Places
  • Tools
  • Research notes
  • Wiki pages
  • Reports

Some of these have been merged -- Places and Tools are now just special wiki pages. Reports were merged into Research notes early on, as they were not well differentiated.

I've been thinking about, and discussing with other Public Labbers, a series of related challenges we face in the Research Note format, and wanted to talk through some of them here in my first post for the newly relaunched Public Lab Blog

One metaphor introduced by Seymour Papert (author of the 1980 book Mindstorms) in the context of educational technologies is the phrase "low floor, high ceiling" -- where a medium with a "low floor" assures a low barrier to entry, but the "high ceiling" simultaneously does not restrict an creator's ability to create complex, powerful works. Mitchel Resnick of the Lifelong Kindergarten group at MIT also argues for "wide walls" in his 2005 paper with Brian Silverman, "Some reflections on designing construction kits for kids" -- which is to say, accommodating a diversity of types of work.

First, let's discuss some of the challenges we've faced with the Research Note format:

Intimidation

Though we've worked hard to make it easy to post a research note, and to lower the technological and cultural barriers to doing so, relatively few of our community of thousands post them. In fact, only around 500 have been posted over five years -- and the rate of publication shown on our stats page is actually down a bit since one year ago.

posts-graph.png

Even the name "Research notes" -- so carefully chosen both to denote informality and to encourage and recognize anyone's contributions as "research" -- has been cited as intimidating: "Is what I'm doing really research?"

Exhaustion

Even our most active members can go weeks or months without posting. Why is this? Is it a problem? Some are busy, sure, but others are actively working on PL projects, but haven't found the time to publish. Have we built up the idea of research notes too much, such that people feel they must be long and carefully crafted? Some folks may "save up" work until they think it's "ready" -- saving time by not pausing their process too often, and waiting until they have something more substantial to share, or they're more assured of the outcome. See, for example, this sampling of four fairly active posters, over the last 52 weeks (the shortest bars indicate a single post):

graph-2.png

Interface

The research note posting interface has been carefully crafted to emphasize simplicity and clarity. But it's definitely not designed for longer-form work. Some on the organizers list and on Github have argued persuasively for a richer, more powerful editor that is simultaneously easier to use without knowing Markdown, the simple formatting system we use.

posting-note.png

Many of the above issues seem to push us in different directions. What I'd like to explore is the possibility of a shorter research note format, perhaps as an alternative in parallel with a longer version.

But first, how do we know what exactly is needed?

How do we know what's needed?

I just want to take a step back here. We've often discussed how to make experiences richer and deeper, because we see the amazing work of our most involved members -- long, articulate posts by @cfastie, @hagitkeysar, and so many others. This is of course a good thing.

But what about everyone who didn't post? I'd like to focus on that hard-to-measure group who we didn't manage to entice into posting something. Basically -- selection bias: we don't have feedback or input from those who aren't participating. If participation were a pyramid, we're only measuring the top, most involved, and I'd like to look at the base -- the lurkers, the observers, and those who we could do better at engaging.

I believe there's a great deal of untapped potential there! Even if we count only the 5-8000 people subscribed to our various websites and lists, that's still over 10x the number of people who've ever posted a research note. And look at the numbers for how many people have posted at least twice, three times, and as many as eight times:

frequency.png

I'd also like to think more about how to better understand that group -- those who aren't posting. I want to think about how to structure a study or survey, for example, that can help us address selection bias and inform the design of our site in a more balanced way. Farm Hack, for example, ran an in-person user study with passers-by (non members) at national organic farming event.

IMG_20150626_111159-2.jpg

Achieving longer form through serial posting of smaller pieces over time.

Short form posting

In thinking about how to reach people who are not yet posting, the idea of shorter, more regular posting is appealing. Rather than "all at once" posts, authors could share (as an example) just their question or background story in one post, their proposed experiment in another, their field test itself in another, and analysis and closing thoughts in a fourth post.

To be clear, what I'm proposing is not cutting down on content, just breaking it up into multiple pieces, and scaffolding that "shorter posts more often" pattern through our editor, which could remain simpler as a result. In fact, as each individual post would need less formatting, the basic posting form could just be plain, unformatted text, as a default. If people could spend more time inviting others into their work, and less time formatting their posts, that seems like a good thing to me.

     question.png

One experiment we did which I think we may be drawing the wrong conclusions about is the Question and Answer function, which was a limited experiment to invite people to make short posts where they ask a question. Although the questions posted are a bit untidy and sometimes oddly formatted, if you look at the authorship of these posts, they're ALL first-time posters! As a format, we hadn't really thought of it as a success, but by this measure it certainly is.

Goals

In summary, although I definitely want to spend time "raising the ceiling," I think we should "lower the floor" as well, and set a goal for ourselves to increase the number of regular posters (i.e. at least one post per month) tenfold in the next year.


17 Comments

This is great to read! I am also excited about the Question feature, and advocate for a direct link to Asking Questions (and Answering Questions) on front of our website. On a personal note, i would post more notes while events are actually happening if i could email in a photo and a subject line (for the title), and some body text.

Reply to this comment...


Hmm, interesting. Do you feel that's much easier than using the smartphone-friendly posting form?

Did you know you can use "Add to home screen" on most phones to make the posting form into an app-like icon? I understand that it's not "the same" as just emailing in. But do you think it's more of an issue of people not knowing they can do this, or that it's really a fundamentally different interaction, or is this more about your specific smartphone interaction preferences?

If we wrapped just the post function up into an app, we could probably make it an Intent, so that people could "Share to Public Lab.org" from the photos app, for example. But this would only work on Android - Apple doesn't allow it.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Great idea - i have just created an icon that goes directly to publiclab.org/post. I will give that a shot for a while and report back. One thing i will keep an eye on is how often (every time? every day?) i have to log in, because logging in on mobile is not so fun, compared to emailing in which presumably would somehow already be connected to my PL account.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Hmm, I don't find that I have to log in that much; perhaps you should open an issue for that, as it should remember you. But yes, that's an extra step it'd be nice to avoid.

Reply to this comment...


Thanks Jeff. As always, thoughtful with an eye for how to broaden participation and make more inclusive. For me, being busy and spread thin is an issue. The shorter mode could help. Pat

Reply to this comment...


Since our inception, looking at and using Markdown has scared off posters-- and I know this because people have been telling me this for four years, in workshops, and I'm not alone. How many people need to bring this feedback up before its treated seriously as actual feedback from real users who don't post, rather than one "question" among many that push us in "different directions?"

We've mitigated this barrier ( the high floor we ACTUALLY have) somewhat with drop-in image uploading and some style guides, but the intimidation factor of either not feeling able or not feeling able to learn to post "nice looking" research notes of the style that more expert Markdown users post is pretty high.

I would disagree that the issues of using Markdown for longer posts and wiki pages mean that fixing this issue with a WYSIWYG editor pushes us in the direction of longer-form content. I think that is a poor framing of the issue. This is merely the framing captured by Github, because it is the way current users understand the problem. It fails to capture the range of feedback we've actually gotten.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


I know I'm going to come off cold and inhuman, but we should not ignore mathematics and known trends.

Most natural works, and human works, exist in some kind of network. As a result of the way networks operate, we tend to see Power Law or Parieto Principle all over; it is often summed as the 80-20 rule. 20% of entities in a network contribute to or are responsible for 80% of something in the network, and 80% of entities likewise represent 20% of the measurement. I think we see exactly that pattern with the number of users who contribute great or small quantities of research notes. Using 80-20 language: 20% of the users contribute 80% of the total research notes, and 80% of the users contribute only 20% of the total research notes. I think this is neither bad nor unexpected. If anything, it demonstrates a healthy network, if not a utopian one ;)

Additionally, I think the post count for four active users is very likely to be well modelled by a Poisson distribution. As told by Wikipedia, such a distribution arises from "a given number of events occurring in a fixed interval of time and/or space if these events occur with a known average rate and independently of the time since the last event".

Clarification of the Poisson process is made by way of example: "For instance, an individual keeping track of the amount of mail they receive each day may notice that they receive an average number of 4 letters per day. If receiving any particular piece of mail doesn't affect the arrival times of future pieces of mail, i.e., if pieces of mail from a wide range of sources arrive independently of one another, then a reasonable assumption is that the number of pieces of mail received per day obeys a Poisson distribution."

So there are two things to ask with that respect: 1) assuming a Poisson distribution of posted research notes, can we decide on average post rate for each user (and is that metric meaningful)? 2) assuming a Poisson distribution is true, why are the note posting times independent from each other? Given the narrative of this research note, one would think notes should be related to each other and thus posting times would be related. Perhaps this is true and the posting distribution is not Poisson. It would be interesting to try to model the post distributions to find out.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Hey Mathew - maybe you misunderstand -- I'm not speaking against a WYSIWYG editor - we've clearly agreed on Github that it's a priority, and my understanding is that we're working to dedicate some staff time towards it in the coming year. A shorter note format could definitely be WYSIWYG.

I agree that WYSIWYG doesn't mean longer, and that we should think of length and ease of interface as separate issues. Given that we've made a WYSIWYG editor project a priority, I'm mostly interested in the question of research note length at this point.

I do think the idea of a "plain text only" version of a post is interesting, though, for high frequency (day-to-day) or first-time posters. What do you think of that concept?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


For plain text only, we might want to look into something like what TwitLonger does. http://www.twitlonger.com/

Twitter begins as just text with 140 characters. TwitLonger breaks the 140 character limit and supports line breaks, but that's about it. It's just text and links.

Since Twitter is accessible, tons of people already use it. I wonder if we could make a way to generate a research note for a user if they send a TwitLonger link to @ publiclab on twitter? Then research notes could be written using someone's favorite long tweet supporting app.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


I don't misunderstand this note, Jeff. It has two sections: a diagnosis of the problem of research note posting, and a proposed remedy-- working on shorter form content strategies. I'm not addressing the question of whether shorter-form content is a good idea right now, I'm addressing the incompleteness of the diagnosis for not taking seriously the idea that Markdown is a user interface issue effecting content posting:

"The research note posting interface has been carefully crafted to emphasize simplicity and clarity. But it's definitely not designed for longer-form work. Some on the organizers list and on Github have argued persuasively for a richer, more powerful editor that is simultaneously easier to use without knowing Markdown, the simple formatting system we use.Many of the above issues seem to push us in different directions. What I'd like to explore is the possibility of a shorter research note format, perhaps as an alternative in parallel with a longer version."

Am I misreading that? it says: 1) the notes are simple and use a simple formatting system right now 2) it isn't designed for long-form work, and we have a persuasive case that WYSIWYG would help that.

Then there's this statement: "But what about everyone who didn't post? I'd like to focus on that hard-to-measure group who we didn't manage to entice into posting something. Basically -- selection bias: we don't have feedback or input from those who aren't participating."

But we DO have some feedback from some people who aren't posting and they've been complaining about Markdown since we started using it four years ago.

After harping on the usability of Markdown for four years, yes, it is now a Github priority on development. But is is actually one? Is it recognized by our Web Team as a priority, as a problem affecting posting? Or is it begrudgingly accepted as something in-demand? if the Web team isn't writing it into their diagnosis of usability issues, I fear it won't be.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


It's absolutely a priority -- so much so, that throughout this note, I've treated it as a given that we're implementing it.

Am I misreading that?

Let me try again on what I mean in that paragraph:

  • we spent lots of effort to strip down the research note form and simplify it
  • both Markdown and our shorter, smaller form are especially bad at long-form work
  • we are implementing a WYSIWYG editor to address the Markdown problem
  • I hope that in all these efforts, we keep ensuring that our interfaces are simple and clear
  • I think a move towards longer form has value but has to be balanced against the advantages of a shorter form

I'm sorry, I didn't mean to just skip over our nearer-term plans here; I'm just looking ahead to what'll come after that. So when I want us to account for selection bias, I mean with regard to longer vs. shorter notes and other issues we'll be facing after the WYSIWYG project. I hope that addresses your concerns!

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Kudos all around for a great site!

  1. I love the Notes and feel guilty I'm in the 80th percentile of lurkers--will have to post something.
  2. Wiki pages are nice but I can't help but think that rather than being the "information super highway" people intended, they are more like "information parking lots" of the internet. There seem to be about 900 wikis? Some have good content but it's hard to find wikis of value unless it's referenced in a Note.
  3. Then what about PLOTS google groups? It's like the legacy Q&A section that I keep forgetting even exists. I think the groups could be reincarnated as a front page Q&A section.
  4. And lastly, another need would be for announcements and events. For example, EPA is holding a live webinar on citizen science air monitoring on July 9th but I don't know how to convey stuff like this.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


I am also interested in a plain text question posting option. Would be good to be able to include pictures though, with an upload method that people understand from other common social media or email standards.

Reply to this comment...


This was brought up several times during the Barnraising also. There were so many awesome people there with great ideas and lots to say, yet I haven't seen them on the Wiki. So we are losing them somewhere. Maybe it's not in the process of posting Research Notes; maybe it's in the process of actually just getting to them? What I kept hearing was more along the lines of "I went to the site, I got overwhelmed/intimidated, I left the site"... I'm not sure they are even getting to the text editor part of it.

Lurkers, I see you out there! Feel free to chime in!

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


When the team I worked with at the Berkeley School of Information was working on our thesis project we looked a lot at how to build intrfacws that encourage easy, low effort contributions to bring someone into a community. One thing we built into rhe platform we developed was modularity on projects. Someone could start a project and add content (such as a Mapknitter map, text, other photos/videos) and they would own that project. But anyone else could request access to the project to submit to it. They might kit have much to add, but it would make them feel more connected than just commenting, since it feels more like a real contribution. Our platform was obviously for a different audience/purpose than public lab, but making it easier for people to collaborate and contribute to each other's work is one option.

Reply to this comment...


User-documentation issues have been THE most challenging problem for just about every grassroots organization I've ever worked with. For projects/campaigns where the primary object was textual/software based, github has always emerged as the best option, even when you factor in the learning curve for non-programmers (and I say this as someone who only recently started identifying as a programmer...). The github format somehow always led to frustration when it came to documenting projects with any kind of physical element however. When my hopes for a "github for hardware" finally did materialize with the likes of Upverter etc... I found the learning curve favored those with professional engineering backgrounds even more than Github does with programmers.

In this context, I found the research note format to be by far the best grassroots, user-documentation platform I have yet used! The key to its success IMHO is in its flexibility. Any single entry can variously serve as a narrative blog post, a Github readme, a step by step tutorial, a journal entry or a discussion forum. On the other hand, I could see how the very flexibility that makes it so useful could also make it difficult to evaluate its effectiveness. I have noticed my posts have been getting less comment activity then a year ago when I could always count on a lively discussion following every new post. Part of this might be related to the separate discussion forums but then again I've noticed activity on almost EVERY listserv-style discussion list I've been involved in has been declining too...

I have no idea if the metrics support this or not, but my point is that the research notes have the ability to evolve in ways that a discussion forum doesn't. So sometimes I'll post a note on something that'll collect dust for months on end and suddenly I'll get an email out of the blue from somebody who saw it and wants to collaborate and the project springs back to life...

In order to ensure the format as a whole continues to evolve, I suggest we take advantage of the power of Markdown!

Speaking as a relative newcomer to the programming world, I like to think I can speak with a certain degree of confidence on what can be considered "accessible" and what is an overly technical interface. So folks will have to trust me when I say MARKDOWN IS NOT OVERLY TECHNICAL!!! Especially, when it is presented in the research note editor! However, so long as we're using it, we might as well take full advantage of its flexibility. This could potentially mean much more than editing from a mobile phone. The cross-platform nature of markdown formatting could be used to fully realize the multi-functional potential the research notes have already demonstrated.

In other words, instead of a note functioning "like a blog" why not allow the content to be reformatted AS A REAL BLOG! I've been experimenting with something along these lines with various markdown-based blogging platforms like GHOST. Unfortunately, hosting requirements proved too much of a headache for me to stick with GHOST. Other platforms like Jekyll, offer a unique solution to the hosting problem by hosting the content in a Github repo via the gh-pages feature. The Ruby-based Jekyll code has always been another barrier for me however, but more recently platforms like hubpress.io have combined the best of both Jekyll and GHOST by providing a node.js framework hosted using gh-pages. Since all the posts would be written in markdown anyway, it would be amazing if we could use some of our research notes as blog posts and vice versa. So if I have a project I would like to share, I would only have to write it once, and the same document could transform into a discussion forum post, blog post, readme file in a code repository etc...

Reply to this comment...


@ajawitz I'm not sure if you have seen the (lengthy) discussion on Markdown in various places, but I can link to at least one part of the longer conversation: https://github.com/publiclab/plots2/issues/151

Very likely we'll be putting a user interface that users can use without knowing or caring how the actual markup text looks. Buttons and that, like word processors that most users are already familiar with. In the backend, we will likely continue storing the markup information as Markdown, which would still support exporting markdown for various other uses. The difference is whether the user must use the markdown directly or use pretty buttons and pretty rendering which hides the Markdown.

There might be a "raw" button that lets advanced users edit the markdown directly if they feel comfortable with it.

I think that's a suitable summary of the current discussion.

Reply to this comment...


Login to comment.