Advice for scientific writing
Denis G. Pelli
Psychology and Neural Science, New York University
April 29, 2013. Added Finding Articles for your Research Project
May 15, 2011. Curled quotes.
February 6, 2011. Added Read it aloud.
January 31, 2011. Enhanced Title to distinguish being motivated by from mere liking.
January 22, 2011. Added “What do I know after reading your paper that I didn't know before?” Added Reviews. Enhanced Take credit. Specify exactly what is the contribution of this paper.
June 30, 2010. Added Show off your data, not the junk.
June 7, 2010. Added Be complete and Take credit.
May 25, 2010. Added Nudging.
January 11, 2010. Enhanced Convince; Keep going; and Communicate. Added items Be friendly; Look at your figures; What is the paper really about?;
Help the reader cite you; and Authors.
Added new section at the end: Avoid common mistakes.
FINDING ARTICLES FOR YOUR RESEARCH PROJECT
Google Scholar. You are probably looking for articles for your research project. There are various ways to look. Google Scholar is my favorite. Type "scholar" into google, and then select "Google scholar". (Or use the link below.) It's a very powerful searcher of all published articles. You may already know of an article, perhaps from the reference list of another article. You can type in part of the reference and ask Google scholar to find it.
Citation. One very nice feature of Google Scholar is the "Cite" button at the end of the article description. The Cite button gives you text, which is a properly formatted citation for your article, in any of three styles. Select APA, and copy that formatted citation to the Reference section of your paper.
NYU's Get-it. When you find an article, through Google Scholar or otherwise, you'll typically get a link to the journal. Some journals will give you the PDF of the article for free. Some want to charge you, typically around $25 per article. That's a lot of money. However, NYU has an extensive collection of paid electronic subscriptions that allow you to get articles for free. To benefit from these licenses, you must go through NYU. You do this by using the NYU Get-It page. You type some citation information for your article into that page and hit Search. It's particularly easy if you know the DOI number for your article. Then you type in (or cut and paste) just that number. Then hit Search and Get-It will find your article and offer you one or more links to get it, free. Typically you'll be asked to provide your nyu id and password. When it works, you'll get to a journal page that allows you to freely download a PDF of your article.
Collaborating on manuscripts, I often find myself repeating the same
recommendations, so perhaps it will be useful to write them down in one place,
where they might be consulted by those who would like some guidance on style.
Some of these comments are meant for students writing their first scientific
essay, and may seem obvious to more experienced writers, but they’re all applicable
to all my scientific writings. These are my opinions, but I learned a lot about
how to make a good figure from John Robson, Beau Watson, and Tony Movshon.
1. Convince. You must convince the reader of an interesting new conclusion.
Interesting, new, and true. If you fail to convince, then
you have no impact and no scientific contribution. Convincing your peers is an essential part of the modern definition of science. It's social not private. By this standard, Leonardo was an artist, because he showed his paintings, but not a scientist because his “scientific” notebooks were published only after his death. If you merely write in a private notebook, or publish in a way that convinces no one, your efforts may be admirable in other ways, but it's not science. What do I believe after reading your paper that I didn't know before?
2. Keep going. Don’t take your audience
for granted. Reading is hard and readers are impatient. They’d rather be doing
their own research. Each sentence must convince the reader to persevere.
One misstep and the reader will put down your paper, probably never to return.
3. Communicate. Ask a friend to read a draft (the whole thing or just the abstract) and tell you the gist.
We often talk of papers being great, as though it were an intrinsic quality
independent of the readers. In fact, papers have value only to the extent that
they succeed in communicating ideas to their readers, the particular audience
that you are trying to reach. Understanding anything new is deucedly difficult.
Thus, what strikes the author’s
ear as perfect may in fact be inferior to a plainer longer exposition that
is easier to read. Polishing should heed reader complaints, especially
what they don’t get. Interrupting the reading with questions, asking
the reader to paraphrase, may reveal what the reader missed when the text fails
4. Be friendly. Most authors would like to publish in Science and Nature,
but the competition is so fierce that only what appear to be earth-shattering
discoveries are accepted. This has the unfortunate side effect of encouraging
authors to write everything as a breathless revelation. Hard things
are presented as though obvious, and, alas, reviewers are too often unduly
impressed by what they do not understand. Personally, I know that it’s
easy to lose me, so I’m grateful and impressed when the author helps
me to understand. Sometimes it’s hard to be both clear and correct at
once, and a comment may help. To ease a hard passage, imagine the reader as
a cherished student, and whisper a stepping stone or gist.
5. Be complete. Another aspect of friendliness is to provide complete results and methods. Nominally we provide just enough to prove our point and allow replication. But it's better science when we provide, within reason, the whole data set and relevant, if not strictly necessary, details. This allows the reader to use our data to address other questions, beyond those posed in our paper. Her question may be sympathetic, critical, or unrelated to our own. This exposure to expert readers helps make science much more reliable than any individual author can be.
6. Take credit. Specify exactly what is the contribution of this paper. The contribution should be specified in the abstract and the conclusion. This is both an obligation to acknowledge and an opportunity to brag. Spell out what this paper should be credited for. Don't be shy. You want the reader to understand the gist, getting the big picture in all its glory. Make it as short, clear, and exciting as you can. Don't hold back, but you must also make clear what is new, the fraction contributed by this paper, by acknowledging what came before. Your conclusion is rarely entirely your own. It usually builds on things that came before. You have an obligation to spell that out so the reader can distinguish what's new from what should be credited to other papers. The reader needs both the gist and the acknowledgment.
7. Draw the figures and put them where they belong. Most of us don’t
want to see the manuscript until you’ve included the figures (graphs,
etc.), but they needn’t be final data. Crude cartoons are fine. We need
to see a graphical expression of what the figure is meant to communicate. (Yes,
please, sketch something now.) Include the figures in the text where they’re
meant to be, not at the end. Some journals still have archaic rules demanding
that the figures be at the end of the submitted manuscript. This is for the convenience of the production
staff at the expense of the reviewers. However, in my experience, the journals
only enforce these rules at the end of the review process, so you can initially
provide an easy-to-read layout for review and, later, once the paper is accepted,
provide the specified layout for production. Number the figures. Every figure should
have a caption explaining what's in it.
8. Look at your figures. Reading graphs is a learned skill. As scientists
become more fluent at reading graphs, they look longer. Try looking at your figure for five minutes. What does it
seem to be saying as an image? Adjust the figure and the text to tell the same story. They are much more convincing when they are in harmony. You
can change anything except the data. These cosmetic changes, over many iterations,
will make your figure much more effective. Multiple graphs should be consistent,
providing a coherent account.
9. Show off your data, not chart junk. The purpose of graphs is not to create a box into which you cram information. The purpose is to communicate with the reader. The essential part of the message is the data. Not the rest of it: boxes, labels, ticks, legends, etc. The rest is necessary, but, compared to the data, it's junk. Don't shrink the data to make room for the junk. The junk should never compete with the data, either for space or for ink. It's like the bride at a wedding. No one should dress so as to upstage her.
10. What is the paper really about? I have several times been stunned, in writing the seemingly superfluous cover letter explaining to the editor
what's so great about the enclosed manuscript, to see ideas emerge that were
not explicit in the paper itself. Borrowing from the letter, the paper was
much improved. It's as though the story is revealed, not invented, and will
tell itself, if we only let it. So I recommend, when you think the paper's done, that you ask yourself what it's really about. The answer may surprise you.
11. Number the pages and the lines. Anything
you send for comment should have numbered pages. It is annoying, when writing comments or a review, to lack page numbers. It’s nice to indicate
the range: “Page 1 of 17”. Remember that many printers
don’t print the top or bottom 0.5 inch. You can make it even easier for your commentator by providing line numbers, from beginning to end. In
Microsoft Word, select
“File:Page setup:Microsoft Word:Margins:Layout:Line
12. Name the file. The file should have a name based on the
title or first author’s name and should end in a number representing
the draft number, e.g. channels3.doc or martelli7.doc. (Some of us have many
manuscripts to keep track of.) Microsoft Word files have a much better chance
of surviving transmission through email if their filename ends in “.doc” or
“.docx”. (Since you’re going
to be emailing the file, it’s best if its name has no spaces or underscores.
Run the words together, or use a dash to separate them, e.g. spatialfrequency.doc,
spatialFrequency.doc, or spatial-frequency.doc.) It’s a good idea to
mention the filename or at least the draft number on the first page, so that one can easily
tell which draft a paper copy represents. Remember to update this draft number every time you create a new draft. Anyone who edits the file
should increment the draft number before sending it to anyone else.
13. Read it aloud. Alex Holcombe mentions “the well-known finding that reading sentences aloud makes it easier to improve their writing. Use of our vocal apparatus manifests the natural articulated rhythms of text, which we might not register when reading silently.” In the brain, there are two streams of auditory processing of speech, one for comprehension and one for articulation (Hickok and Poeppel 2007). Reading aloud uses both.
14. Help the reader to cite you. There are many kinds of document today, e.g. blogs, and, if you want
to be cited, it’s best to show the reader how to cite your
work. Manuscripts should have a title and authors
even when they are tentative and subject to revision. This is part of laying
claim to an idea. Put enough on your front page that someone
who receives it could cite it. The key things are title, authors, and date.
Until publication, the title and authors are subject to revision, but without
them, and the revision date, the document is almost impossible to specify.
15. Nudging. Collaboration is wonderful. The key ingredient is that
you both must need each other. That's what will carry you through the hard
patches. However, many manuscripts die sitting on the desk of someone
to get back to it soon. How do you get it moving again? This is often described
in moral terms specific to the personal association, but, after many years of
sending and receiving reminders, I've come to think that it's a professional skill.
Some people are good at it and they collaborate to produce many papers. Others aren't immoral; they just aren't good at it. Watching,
from both sides, what works and what doesn't, I note that there is a trick
to it. Start very very mildly, lightly reminding. And stay there. Don't escalate.
This is counter-intuitive because, as a sender, one is embarrased by the implicit
criticism of the reminder, and one feels a need to justify the action by moralizing
and describing dire consequences. But all that negative stuff discourages the
probably needs only the reminder and perhaps some encouragement. And, of course,
do it. Always very
lightly, but frequently enough to keep the paper in your recipient's mind.
Mastering this unsung skill and collaborating with good nudgers — nudging and being nudged — may greatly increase the number of papers that you publish.
16. Reviews. Receiving reviews, from the journal editor, that don't recommend acceptance is distressing. They may seem stupid and mean. I suppose some are, but mostly they are the best attempts of people just like us, unpaid volunteers, struggling to understand the paper and give sound advice to the journal editor and the authors. Very slowly, over the years, I've been learning how to read reviews. It's hard, but very useful, to spend enough time with the review to get to the point where you can identify with the reviewer and imagine writing what is before you. When this succeeds, I suddenly realize that, oh, yes, if I were the reviewer and I assumed that then I would expect this, not what we found, and be disappointed or even dismayed by what we said. At that point, with a viable model of the reviewer in me, I can consider how we might say things differently to include the reviewer and keep her on the path of our story. If this reviewer got it wrong, it's likely that other readers might too. Fixing it for the reviewer might grealy increase the number of readers who accept our conclusion. Sometimes it's just a matter of adding a few words, acknowledging a consideration or contrary assumption, or cautioning against a tangential attractive nuisance. Negative reviews are very hard to read, but the editor did pick the reviewers to be knowledgeable experts, so they are a sample from our target audience and are thus an invaluable guide for how to increase the success of our paper in communicating. Even if it's wrong, the negative review is evidence of faulty communication that wants repair. If I can put myself in his or her shoes, I have a chance to whisper words into my paper that might bring him or her on board.
Get an article
The Oxford English Dictionary
Title. One usually thinks of the title as a statement of scope or a memorable gist, inviting the reader and reviewer. However, note that when choosing what paper to cite, wrters will often choose a title that matches the point they are trying to justify, so that a concrete assertion (a sentence) may garner more citations than a generic topic (a subject). Of all the phrases in your ms, it is the title that has the greatest effect on editors, reviewers, and readers, so it's worth getting your friends to judge it, especially when you're trying to choose among several candidates. I've always asked, “Which do you like best?” but I've just discovered (in 2011) that this is the wrong question. Today, my friend liked the short sober title better, but, when asked which title was better at convincing her to read the abstract, she chose the long cutesy title, by a “large” margin. For getting published and read, convincing the reader to proceed is what matters, so, henceforth I'll ask about that instead of liking.
Authors. There are various widely used but inconsistent principles
for ordering the
authors’ names in the byline. I recommend strict descending
order of contribution, but you should
consider the expectations of your audience. Neuroscientists often put the student
of how you order the names, all the societies encourage you to specify, in
what each author contributed. I now do that in all my papers. My favorite article on this topic is Riesenberg,
D., & Lundberg, G. D. (1990) The order of authorship: who's on first? JAMA,
264(14), 1857. Several important societies comment on current practice: APA (8.12
Publication Credit), Society
for Neuroscience (1.3.3), PNAS, and ICMJE (Uniform
Requirements for Manuscripts Submitted to Biomedical Journals).
Abstract. Convince the broadest possible audience that this is interesting
and important. In a few sentences, tell us what you did and what you conclude.
Bear in mind that most people considering whether to cite this paper will assume
that all the conclusions are present in the abstract. It is my impression that most citations, today, are based on reading just the abstract. The main measure used today for scientific impact is still the citation count. Thus, to some extent, like it or not, for assessment purposes, your scientific contribution is your abstract not the paper. So consider how well your abstract holds up on its own.
As noted above: Specify exactly what is the contribution of this paper. What will the reader believe after reading your paper that she didn't know before?
Introduction. The main purpose of the intro is to motivate the work (i.e. convince the reader that this question is sufficiently interesting to be worth reading about), but this is also where you credit what’s already been done by others, especially by potential reviewers. The intro typically takes the form of a historical review, but that’s more a pretext than a purpose. The purpose is to motivate and give credit. If you are not yet well-known in this field it may be important to the reviewers that you show awareness of the key papers in the field. You can get this list by scanning the introductions of other papers. You needn't praise. It's enough to cite.
Methods. It’s important that this be correct, complete,
and understandable. It should enable the reader to replicate your experiments.
The writing can be ugly and repetitive, but it must be complete.
How will you feel if someone fails to replicate your result uh oh! because
you omitted an important detail? It is usually skipped (or lightly skimmed)
in a first reading, and consulted later, to look up details. The dominant tradition is to place methods in the middle of the paper. Experienced readers skim or skip methods on a first reading. The high-impact journals tend to relegate methods to the end. I'm coming to think that all journals should do that, so we wouldn't have to skip. Methods belong at the end.
Results. Data. Graphs. The results text should
have a very plain style. “Just the facts, ma’m.” Only minimal interpretation
and comparison to other work. But do mention replication and inconsistencies
(real or merely apparent) with past work. Sometimes the empirical result
is more or less the conclusion of your paper. Sometimes that conclusion needs
a reasoned argument, which may appear here or in Discussion.
Discussion. Try to give the reader the big picture. Take a step back. Try to forget your stake in this and guide the reader through your garden, noting the various considerations, positive and negative, that seem relevant. Connect this work to that of others. Even distant connections help, as readers come from various places and it always helps to understand the connection, however distant, of what’s new to what’s familiar. However, the meandering connections, desirable as they are, are no substitute for a tight argument that forces the reasonable reader to accept your conclusions. Ultimately that’s the core of your contribution.
Conclusion. Most papers published in psychology
do not have a final section labeled “Conclusions”. My own view
is that it is rarely reasonable to publish a scientific paper without a conclusion,
and that it is helpful to draw attention to its presence by setting it off
in its own section. The conclusion should be short and as strong as you can
make it. I consider the conclusion to be the reader’s reward. This is where you deliver on the first item, above: What do I believe after reading your paper that I didn't know before reading it?
Acknowledgments. There’s a lot of freedom here, but try to be
concrete (what exactly was the contribution?) and flattering. People should
be glad to be thanked. If you can’t word it to achieve that effect then
don’t bring it
up. Specify what each author contributed to the paper (see Authors, above).
References. It is almost impossible to type references by hand without introducing errors. This is something that computers are good at. Use Endnote (available for Mac and Windows). Endnote can download references to articles directly from PubMed (the National Library of Medicine) and PsycINFO and can download references to books from the Library of Congress and most university libraries, including NYU. Endnote will format your references in whatever style you like. (For PsycINFO you’ll need an NYU-specific Endnote connection file.) The resulting reference list is very accurate, and can optionally include a link to the PubMed abstract.
Future research. No! Scientific papers rarely provide “suggestions
for future research,” for good reason. Authors usually do not want
to share their best unexploited ideas, and it’s disingenuous to recommend
less-than-best ideas. High school students and undergraduates, writing their first paper, often use this section to trash their own work, explaining how much better the paper might have been with more time and 20-20 hindsight. This “might have” stuff has no place in your paper. You should be reporting only what you actually did, and drawing conclusions from that. Your readers do not want to hear about what you might have done. It's boundless and boring. They don't have time for that.
Vector graphics vs. pixel-based images. For production, all journals
now strongly recommend vector graphic files instead of pixel-based files. They're
right. This makes your PDF reprint compact and suitable for printing at any
resolution. However, those considerations are irrelevant at the review stage.
For review, it is best to provide the figures inside your manuscript text document,
usually Microsoft Word, more or less as they would be placed in the final publication.
This makes it easier for the reviewers to read your paper, which is very important.
Reviewers don't care whether you have vector or pixel graphics, provided the
resolution is high enough for everything to look sharp. That's good because
Microsoft Word makes it very hard to achieve vector graphics within your Word
document. Word will import many vector formats, including PDF, but mulishly
displays only a blurry pixelated rendition. The best solution I've found is
to open the PDF in Adobe Illustrator and export it as a high-resolution pixelated
PNG image (e.g. 600 dpi). Word imports and displays PNG images faithfully.
This results in a Word document with sharp pixelated images, perfect for reviewers.
When your paper is accepted, the journal will ask you to provide vector graphic
files for production, one file per image. You can send them your PDF files.
Kaleidagraph and Word. It's hard to get a graph from Kaleidagraph into
Microsoft Word and have it look sharp when you print your Word document. Here
are two approaches that I recommend: the first is quick and dirty (not sharp);
the second takes many more steps but makes graphs in Word as sharp
as you like.
Copy & paste. The quickest way to get your graph from Kaleidagraph
into Word is to use the Kaleidagraph “Edit:Copy graph” menu item
and then paste that into your Word document. This works, and may be acceptable
for early drafts shared between authors, but the image is usually a bit blurry,
not the crisp sharp image you'd like your reviewers to see.
Print to PDF. Ask Kaleidagraph to print to PDF. The PDF file is a vector
graphic, crisp. Great.
Crop the PDF. Print to PDF produces an 8.5x11 image. If you're going
to insert this image into a document, it's nice to crop the file down to just
your image. Open your PDF file in Adobe Acrobat Pro. Select &ldquot;Document:Crop
pages&rdquot; to get a cropping panel. Enable the radio button &ldquot;Remove White
Margins&rdquot;. Click Ok. Close the document; click Ok to save changes.
Don't insert PDF into Word. In Word, you can select the menu
file&rdquot; to import the PDF. However, even though the PDF file is a vector
graphic, Word mulishly renders it as a blurry pixelated image. Yuck. Don't
Convert PDF to PNG. Use Adobe Illustrator to open your PDF file.
Use Illustrator File:Export:PNG and set a high resolution (600 dpi) so the
image will later look sharp in Word. The exported PNG file is pixelated, but
has enough resolution to look sharp. (Avoid the export option called &ldquot;Save
for Microsoft Office&rdquot;, which inexplicably gives you about half the resolution
you want, so everything looks unpleasantly soft.)
Insert PNG into Word. In Word, select the menu item &ldquot;Insert:Picture:From
file&rdquot; to import the PNG file. Word renders PNG images faithfully, so a
high-resolution image will be sharp.
Error bars. Usually every plotted point representing a measured value
should have error bars designating a 95% confidence interval. Usually that
corresponds to ±1 standard error. Please omit the hats that many plotting
programs add to the end of each error bar. The hat adds clutter, making it
harder to see the data.
Caption.Every figure should have a caption, beneath the figure, explaining
what’s in the figure. Usually the caption begins with a figure number,
a title, and a description of the horizontal and vertical scales. I prefer
the ordinary English words “horizontal” and “vertical” over
the jargon names “X” and “Y”. If your figure includes
someone else’s data or is based on their figure, cite the source at the
end of your caption.
Figures. All text within a figure should be in Helvetica (or Arial),
including the axis labels, etc. (The figure caption should be in 10 point
Helvetica, to help distinguish it from the rest of your text.) Don’t
use bold. Capitalize only the first letter, as in a sentence, e.g. Spatial
In general, remember that the graph is meant to express the data, and that
the data themselves should draw the most attention, like the bride at a wedding.
The rest of the stuff (scales, labels, legends) should recede into the background,
not compete for attention. I usually like to represent data as points and
the model as a solid line. We usually use Kaleidagraph (available
for Mac and Windows). We typically use logarithmic scaling. It’s not
an absolute rule, but I find that papers are easier to follow if a log unit
has a consistent length (e.g. 1 inch) within a graph (horizontal vs. vertical)
and throughout the paper. In Kaleidagraph, you set the length of the X and
Y axes by selecting Plot:Set Plot Size:Axis size:. If you do display error
bars, omit the distracting hats at the ends of the error bars. When you send
the final graphics file to the publisher, vector graphics are preferable to
pixelated images, because they look better and take up less space in the final
PDF file for your published paper. Kaleidagraph’s export options are
poor, but you can copy to clipboard a PICT with Postscript, and, if you’re
using Mac OS X or have Acrobat installed, then you can Print to PDF. The latter
is the best way to produce files for production of your article by the publisher.
When you email any graphics file, the filename extension matters: .jpg, .gif,
.pdf. For any other file I suggest that you enclose the files in an archive
(e.g. zip or stuffit) to protect the file resources, which will otherwise be
stripped in the course of emailing.
Symbol size. Some graphics programs, including Kaleidagraph, have poorly
matched sizes of symbols. Obviously, a square and a diamond (the square rotated
45 deg) should have the same area to match visually, but Kaleidagraph matches
them in width, so the diamond is too small for a visual match. Look at your
symbols and adjust the sizes to achieve a visual match.
Color. Some figures, e.g. equiluminant stimuli, demand color. Some
figures may benefit from color, but don’t really need it. Most of your readers don’t
have a color
printer, so, if possible, design your figures to be completely understandable
in a black and white print out. Distinguish symbols and lines by shape and
dashing, and refer to them, in the text, by those achromatic properties.
Helvetica. As of 2007, none of the available versions of Helvetica
are adequate for scientific use. The Helvetica provided by Apple with Mac OS
X lacks italics. (You need italic to correctly represent
mathematical variables. Some programs, e.g. Word, will let you fake an italic,
by slanting, but Adobe Illustrator won’t. ) The versions of Helvetica
sold by Adobe and Linotype lack unicode support. Unicode extends ASCII to a
16-bit code, allowing us to specify any character shape (glyph) independent
of the font. For scientific text this is very helpful because it allows you
to include Greek symbols without changing font. However, unicode is still quite
new and neither Adobe’s nor Linotype’s Helvetica supports unicode.
Helvetica does, but the lack of italics rules it out. Use Arial instead.
Arial. The full history is complicated, but, in effect, Microsoft created Arial by copying Helvetica, to save money. The only difference
I notice is that the “1” in Arial has a longer diagonal line. Type
designers notice other subtle differences.
can test your
ability to distinguish them. (Thanks to Hannes Famira for these links.) The
version of Arial provided by Apple in Mac OS X lacks unicode support. However,
Microsoft Office comes with a better version of Arial, which does support unicode
and includes the Greek characters. So, on your Mac OS X computer, you should
and replace it with a copy of:
Applications/Microsoft Office 2004/Office/Fonts/Arial
Heading and labels. With others, I helped convince Journal of Vision to adopt sentence-style capitalization for all headings and figure labels. Capitalize only the first letter of the first word.
Numbers. When presenting numbers smaller than one, the
decimal point should always be “covered,” so you should replace “.1” by “0.1”.
The problem is that the printed decimal point may be so tiny as to disappear.
If you put a leading zero in front of it, the reader will still know that
Units. Physical measures, e.g. “10 ms”, should always
include a space between the number and the the word(s) specifying the units.
Equations. Don’t confuse mathematics (equations) with computer programming (assignment statements); they have different rules. Here are my suggestions for equations. Use MathType (available for Mac and Windows). Math variables, like E and x,
should be italic and only one letter long. Be friendly to your variables,
don’t set them off by commas. Don’t use “*” to mean multiply.
Don’t use multi-letter variable names; long names are common in computer
programming but confusing in math, where multiplication is implicit,
as in ax. However, you can use a long text subscript, as in Lbackground.
Subscripts that are not variables should not be in italic, eg crms
Functions, like sin and log, are not variables and should not be italic.
Avoid the temptation of indicating an approximate value by “~”, as most journals print that symbol almost indistinguishably from the minus sign “ ”, which is likely to confuse your readers. Instead of “~” use “about”, “roughly”, “approximately”,
or the approximately equal symbol “” that
has two wiggly lines, not one.
Don’t underline; reserve that for links. Underlining is a proof reader’s mark indicating italic,
which was incorporated into typewriters because they couldn’t do italics.
It was not meant to appear in printed material. Some journals allow it, but
I think it looks bad. In any event, it has become a fairly standard way of
indicating a URL (a web link), so I suggest restricting it to that role.
I also suggest never using the underscore character “_”, especially
in filenames, because if you make that text a link then it will be underlined,
and, once underlined, an underscore is indistinguishable from a space.
Technical tip from Kaleidagraph support: Using font characters as markers. You
can’t modify the markers that are built into KaleidaGraph, but you can
add text error bars to your plot. Text error bars are normally used to annotate
the points in the plot. You could create a Line plot, hide the original markers,
and add text error bars to the plot. As an example, you might create a text
column with the letter a in each row. Once you add this column as text error
bars, you can double-click the text and change the font to Zapf Dingbats or
some other font that has different symbols in it. The manual and help file
contain information on adding text error bars to a plot. To use them as markers,
you would want to add them as X error bars. You would also want to make them
single-sided and have the text centered (using the Center Text option).
Avoid common mistakes
1. Don’t apologize for good work. In
my experience, it is common for students writing their first scientific paper
to end their discussion with a devastating self critique, pointing out that,
with enough hindsight, skill, money, and time, everything could have been done
better to reach a stronger conclusion. So what? Usually this whole self-scourging
paragraph should be deleted. Research has two kinds of limitation—you
could have done better and your conclusions are qualified—and only the
latter should be reported in the scientific paper. (And that report of qualifications
should be brief, just enough to let the reader know.) The reader needs to know
the qualifications of the conclusions of the actual study, but this need is
not served by knowing what the experimenter could have done differently. What
matters is what was actually done. Do the results warrant the conclusion? In
a similar way, conclusions are few and worth telling, but the list of things
that you cannot conclude is endless, so skip it. Needless apologies for what
you cannot conclude or could have done better may worry the reader and undermine
her faith in your words, canceling your scientific contribution. Self criticism
is rare in scientific papers, for good reason. There is a huge supply of manuscripts.
Reviewers are hard-pressed to keep up, trying to separate the wheat from the
chaff. Reading that the author believes the work to be deficient may convince
the reviewer that this manuscript is less worth reading than others and should
2. State limitations laconically. Make
your conclusion as strong as possible, and no more. Its limits must be clear,
but harping on those limits will dismay the reader, shaking her confidence.
Everyone knows that increasing the sample would reduce the standard error.
That goes without saying. As authors, we know best what we actually measure.
Our extrapolations beyond that, to other populations and conditions, depend
on linking assumptions. This is essential to every scientific study and rarely
3. Former intentions don't matter. Sometimes
experimenters measure one thing, thinking that they are measuring another.
Even without mistakes, experimental results often provide a more compelling
answer to a new question, different from the original motivating question.
In any case, the experimenter's former intention is irrelevant to the scientific
report of the results. What matters is what happened, regardless of what the
author was thinking. In scientific papers, the actual history of the author's
thoughts is usually suppressed in favor of a fictional history that streamlines
the argument leading to the final conclusion. In motivating the reader, use
your best current understanding, unconstrained by your original intentions
and motives. The point of the paper is to convince, not to recount. It's science,
4. Significance versus statistical
significance. An undergraduate student
asked me, “If a study does not produce significant results, why would
it be published?” To answer that, we must distinguish two kinds of significance.
The ordinary sense of “significant” (or “insignificant”)
is that something is meaningful and important (or not). “Statistically
significant” (or “statistically insignificant”) means that
we can reject the null hypothesis (or not). These are different things. Showing
statistical significance does not establish importance. Furthermore, showing
that an effect is statistically insignificant can be important.
Usually we only care about big effects. If twice the standard error is small,
a statistically insignificant effect is either small or absent. It’s
not big. Concluding that the effect is negligible is often an important finding.
5. t-test. Statistical significance
is often assessed by a t-test, i.e. whether the sample means differ by more
than two standard errors. I’m looking for a short tutorial that would
explain this to students who may know standard deviation, but are unfamiliar
with standard error. Here’s a link to wikipedia, but it seems longer
and harder than it needs to be. I’m looking for something better. Suggestions?
6. Power: How big should the sample be? I was surprised
when this issue arose in the essays of my 2010 undergraduate seminar “Experiments
in Beauty”. Here are quotes from five of the ten essays (mostly
two-authored). All make a similar comment about the sample size:
study consisted of an unusually small sample size relative to standard psychology
experiments. Statistical analyses are generally intended for sample sizes
of at least thirty participants, thus a greater population may have produced
It is true that our in-class experiments used around 8 observers per
group, much less than the 20 or 30 that is typical of many psychology studies.
However, all of these comments are wrong in presenting this as a flaw. We need
a sample that is big enough to yield a strong conclusion. Statisticians call
this “power”. We need enough samples to attain enough statistical
power to decisively answer the question that we pose. Testing more participants than we need would be a waste of effort, needlessly discouraging us from addressing new questions. In our class, over the semester, we did eleven experiments, each in an hour. There’s no time to waste. We usually
compare the sample means of two groups exposed to different conditions. When
the two sample means differ by at least two standard errors, we reject the
null hypothesis that the groups are identical. The only reason to increase
the sample size is to reduce the standard error, increasing the power. There
is no other benefit. Some of the experiments yielded effects that were statistically
significant: a difference between sample means of at least 2 standard errors.
Other experiments yielded differences that were less than two standard errors,
statistically insignificant. Both results are useful. With more samples the
standard error would be reduced and a small effect that formerly failed to
reach significance might become significant. However, never forget that achieving
statistical significance does not establish importance. No study can ever say
that there is no effect. It can only put an upper bound on how big the effect
might be, namely no bigger than two standard errors (with 95% confidence).
Increasing the sample size in our experiments would not change that basic fact.
It would merely reduce the standard error. In most of our experiments, the
observers were rating beauty on a scale of 0 to 10 and, with about 8 observers
per group, the standard error was about 0.5 beauty points (on that 10-point
scale). To be statistically insignificant, the effect must be smaller than
two standard errors, i.e. less than 1 beauty point in most of our experiments.
In this class we were looking for and found big effects: 1 or 2 beauty points.
We don’t care about small effects,
less than 1 beauty point. The statistically insignificant effects were trivial,
smaller than 1 beauty point, practically no effect. That’s a strong conclusion.
Increasing the sample size would reduce the standard error, but we’ve
already ruled out the possibility of a big effect, so there is no reason to
run a bigger study.
b. “If our study were to be replicated, ... First, we
would obtain an adequate representative sample size of at least twenty participants
c. “The results of our study are limited, ... We
... only had ... eight subjects per group, which is a small sample from which
to make generalizations.”
d. “Another possible future research project
would be to perform this original experiment on a larger sample, ... With
a larger sample size, one can make more broad and general conclusions about
the original hypothesis of this study.”
e. “It is crucial to test a
larger pool of participants ...”
7. What population are we talking about? This
too arose as an issue in the final essays of my 2010 “Experiments in
Beauty” seminar. Three of the ten essays note that their sample is unrepresentative
of the US population:
a. “In addition, our subjects constituted an unrepresentative
sample; the majority of subjects were female psychology students at New York
These admissions of limitation all make the same implicit assumption. They set the goal
to be conclusions about the general US population. Our class is, indeed, unrepresentative
of the US. But that’s an absurd goal. The National Institutes of Health, supported
by US taxes, is obligated to improve the health of all Americans and encourages
its grantees to follow suit, but, in this class, we have no such obligation.
We can study any group we like, and, for practical reasons, it behooves us
to study the class itself. Having formed strong conclusions about them, we can speculatively
generalize to larger populations, e.g. female psychology majors at NYU,
US undergraduates, beauty scholars, or the whole human race. It’s perfectly fine to study just a small group. Scientific
conclusions about even just one individual are of great value. Much of neurology
is based on case studies, careful study of a single individual leading to strong
conclusions about that person.
b. “... a stronger result may be determined by replicating
this experiment with various changes. First, the subject pool in this experiment
consisted of predominantly female NYU undergraduate psychology students. This
sample of students may not necessarily generalize to the entire population.
It is crucial to test a larger pool of participants, ...”
research study comes with strengths and drawbacks. ... Since the participants
were all psychology students with interest in beauty, that may have contributed
to skewed results as they do not accurately represent the general population.
... Possible directions that future research studies can take would be to try
and have a larger sample size that is representative of all people across America.”
Thanks to Melanie Ceder, Angel Patel, Aretha Soderstrom, Diana Balmori, and
Cesar Pelli for helpful suggestions.
• Working around bugs in Microsoft Word
• Literature search tips
to the top