The trouble with genomic sequencing, is that it is too cheap. Anyone that has a bit of extra cash laying around, you can scrape the bugs off your windshield, sequence them, and write a paper. Seriously?

Yes, seriously now: as we sequence more and more genomes, our annotation tools cannot keep up with them. It’s like unearthing thousands of books at some vast archaeological dig of an ancient library, but being able to read only a few pages here and there. Simply put: what do all these genes do? The gap between what we do know and what we do not know is constantly growing. We are unearthing more and more books (genomes) at an ever-increasing pace, but we cannot keep up with the influx of new and strange words (genes) of this ancient language. Many genes are being tested for their function experimentally in laboratories. But the number of genes whose function we are determining using experiments is but a drop in the ocean compared to the number of genes we have sequenced and whose whose function is not known We may be sitting on the next drug target for cancer or Alzheimer’s disease, but those proteins are labeled as “unknown function” in the databases.

The red line is the growth of protein sequences deposited in TrEMBL, a comprehensive protein sequence database. The blue line illustrates the growth proteins in TrEMBL whose function is know, or at least can be predicted with some reasonable accuracy. The green line is the growth in the proteins whose 3D structure has been solved. Note the logarithmically increasing gap between what we know (blue) and what we do not know (red). Image courtesy of Predrag Radivojac.

Enter bioinformatics. CPU hours are cheaper than high throughput screening assays. And if the algorithms are good, software can do the work of determining function much cheaper than experiments. But therein lies the rub: how do we know how well function prediction algorithms perform? How do we compare their accuracy? Which method performs best, and are different methods better for different types of function predictions? This is important because most of the functional annotations in the databases come from bioinformatic prediction tools, not from experimental evidence. We need to know how accurate these tools are. Think about it this way: even an increase of 1% in accuracy  would means that hundreds of thousands of sequence database entries are better annotated, which in turn means a lot less time in the lab or in high throughput screening labs going after false drug leads.

So a few of us got together and decided to run an experiment to compare the performance of different function prediction software tools.  We call our initiative the CAFA challenge: Critical Assessment of Function Annotation. There are many research groups that are developing algorithms for gene and protein function prediction, but those have not been compared on a large scale, yet. OK then: let’s have some fun. We, the CAFA challenge organizers, will release the sequences of some 50,000 proteins whose functions are unknown. The various research groups will predict their functions using their own software. By January 2011 all the predictions should be submitted to the CAFA experiment website. Over the net few months, some of these proteins will get annotated experimentally. Not many, probably no more than a few hundred judging by the slow growth of the experimental annotations in the databases. But we don’t need that many to score the predictions. A few dozen will do.

On July 15, 2011 we will all meet in Vienna, and hold the first-ever CAFA meeting as a satellite meeting of ISMB 2011. This will be the fifth Automated Function Prediction meeting we have been holding since 2005. Only this time, there won’t just be the usual talks and posters, there will be the results of a very interesting experiment. The International Society for Computational Biology is generously hosting our meeting, and judging by the response we are getting so far, we will need one of the larger halls.

Learn more at http://biofunctionprediction.org If computational protein function prediction is your thing, join the CAFA challenge. If you are just an interested observer, keep an eye on the site. In any case, please spread the word.  Finally, if your company wants some publicity, get in touch! We could use the sponsorship ^_^

Acknowledgements: I would like to thank the CAFA co-organizers, Michal Linial and Predrag Radivojac. The CAFA steeering committee: Burkhard Rost, Steven Brenner, Patsy Babbitt and Christine Orengo for supporting us, keeping us on the straight and narrow and for incredibly useful and insightful suggestions.  Sean Mooney and Amos Bairoch for hashing out the assessment.  Tal Ronnen-Oron and the rest of Sean Mooney’s group for setting up the CAFA website. The International Society for Computational Biology for sponsoring us. The community of computational function predictors that have participated in and supported past meetings on computational function prediction, the research groups that have registered to CAFA so far, and those that will register soon   Finally, Inbal Halperin-Landsberg for coining the name CAFA. I apologize in advance if I left someone out.

GO CAFA!

Fast forward to July 1, 2011: the meeting is so on. We have 38 teams participating in the challenge, with over 50 algorithms. We have about 600 proteins whose functions have been predicted. We have a great line up of speakers including Janet Thornton the Director of the European Bioinformatics Institute and Amos Bairoch, the founder of SwissProt. The full program is available. The Radivojac, Mooney and Friedberg labs have been working very hard, and we do have some really great results to report, and some surprising activities. We are also grateful to the NIH and the US Department of Energy for their support.

So if you are planning on being at ISMB this year, drop by  the AFP/CAFA SIG. We are messing with something brand-new, which is what science is all about.

Miami University has  joined the National Genomics Research Initiative (NGRI) offered by HHMI Science Education Alliance (SEA) in their Phage Genomics course. The students go directly into the lab, participating in an authentic research experience. In a full-year academic course they:

• isolate and characterize bacterial viruses from their local soil
• prepare the viral DNA for sequencing
• annotate and compare the sequenced genome

The Department of Microbiology at Miami University is offering this course in the upcoming year: four of our faculty will be teaching it. Phage isolation, electron microscopy, DNA sequencing in the first semester, annotation and comparative genomics in the second. And I get to teach the bioinformatics bit: annotation and comparative genomics. Woo-hoo! The great thing about this course, is that unlike most lab courses, the students (and faculty) will be setting up experiments intended not only to teach, but also to discover something new.  Also, the results of the research are meaningful. Genomics data generated by student participants will be used by other researchers to answer medical, ecological, and evolutionary scientific questions. Bacteriophages (viruses that infect bacteria) affect the biopsphere so profoundly, it is almost impossible to imagine. Their sheer biomass is equal to that of 75 million blue whales, and marine bacteriophages  kill about half of marine microbes every day. Bacteriophages have a huge host range, mind-boggling number of particles in the biosphere (10³⁰) and, above all, the genetic diversity is unmatched by all other life combined. Participating students will see how their data may be used by other researchers in the SEA network — truly collaborative, crowdsourced science. Here are the genomic sequences of SEA-sequenced bacteriophages already in GenBank.

If you are an incoming Miami freshman, and want to jump in and do some real science, it doesn’t get much better than this. Check out our course page, and ask about the Bacteriophage Biology course in your orientation. You may even get your name on a paper like these students from other participating universities.

Pope, W., Jacobs-Sera, D., Russell, D., Peebles, C., Al-Atrache, Z., Alcoser, T., Alexander, L., Alfano, M., Alford, S., Amy, N., Anderson, M., Anderson, A., Ang, A., Ares, M., Barber, A., Barker, L., Barrett, J., Barshop, W., Bauerle, C., Bayles, I., Belfield, K., Best, A., Borjon, A., Bowman, C., Boyer, C., Bradley, K., Bradley, V., Broadway, L., Budwal, K., Busby, K., Campbell, I., Campbell, A., Carey, A., Caruso, S., Chew, R., Cockburn, C., Cohen, L., Corajod, J., Cresawn, S., Davis, K., Deng, L., Denver, D., Dixon, B., Ekram, S., Elgin, S., Engelsen, A., English, B., Erb, M., Estrada, C., Filliger, L., Findley, A., Forbes, L., Forsyth, M., Fox, T., Fritz, M., Garcia, R., George, Z., Georges, A., Gissendanner, C., Goff, S., Goldstein, R., Gordon, K., Green, R., Guerra, S., Guiney-Olsen, K., Guiza, B., Haghighat, L., Hagopian, G., Harmon, C., Harmson, J., Hartzog, G., Harvey, S., He, S., He, K., Healy, K., Higinbotham, E., Hildebrandt, E., Ho, J., Hogan, G., Hohenstein, V., Holz, N., Huang, V., Hufford, E., Hynes, P., Jackson, A., Jansen, E., Jarvik, J., Jasinto, P., Jordan, T., Kasza, T., Katelyn, M., Kelsey, J., Kerrigan, L., Khaw, D., Kim, J., Knutter, J., Ko, C., Larkin, G., Laroche, J., Latif, A., Leuba, K., Leuba, S., Lewis, L., Loesser-Casey, K., Long, C., Lopez, A., Lowery, N., Lu, T., Mac, V., Masters, I., McCloud, J., McDonough, M., Medenbach, A., Menon, A., Miller, R., Morgan, B., Ng, P., Nguyen, E., Nguyen, K., Nguyen, E., Nicholson, K., Parnell, L., Peirce, C., Perz, A., Peterson, L., Pferdehirt, R., Philip, S., Pogliano, K., Pogliano, J., Polley, T., Puopolo, E., Rabinowitz, H., Resiss, M., Rhyan, C., Robinson, Y., Rodriguez, L., Rose, A., Rubin, J., Ruby, J., Saha, M., Sandoz, J., Savitskaya, J., Schipper, D., Schnitzler, C., Schott, A., Segal, J., Shaffer, C., Sheldon, K., Shepard, E., Shepardson, J., Shroff, M., Simmons, J., Simms, E., Simpson, B., Sinclair, K., Sjoholm, R., Slette, I., Spaulding, B., Straub, C., Stukey, J., Sughrue, T., Tang, T., Tatyana, L., Taylor, S., Taylor, B., Temple, L., Thompson, J., Tokarz, M., Trapani, S., Troum, A., Tsay, J., Tubbs, A., Walton, J., Wang, D., Wang, H., Warner, J., Weisser, E., Wendler, S., Weston-Hafer, K., Whelan, H., Williamson, K., Willis, A., Wirtshafter, H., Wong, T., Wu, P., Yang, Y., Yee, B., Zaidins, D., Zhang, B., Zúniga, M., Hendrix, R., & Hatfull, G. (2011). Expanding the Diversity of Mycobacteriophages: Insights into Genome Architecture and Evolution PLoS ONE, 6 (1) DOI: 10.1371/journal.pone.0016329

For the record, I do not consider myself to be a remarkable speaker, and I can use improvement. However, I do not think I am a poor one either. These tips are not primarily for you, they are for me. They are basically a collection of dos and don’ts which bubbled up over the past few weeks. Most are obvious and have been talked about by many others. Some are less obvious. As this tips collection is for myself, they are very eclectic, just like your presentation should not be.

1. Know your audience. Who are you going to give a talk to? Then tailor every bit of your presentation to that audience. If you are speaking to a crowd of bioinformaticians, you probably do not need to introduce MrBayes, or what HMM stands for and what it is good for. If you are talking to experimental biologists, then yes, you do.On the other hand, do not dwell on common molecular biology assays. Your goal here is to be clear and engaging. We all know that visiting seminar speaker who is coming from a tangentially-related department (i.e. a biophysics speaker in a molecular biology department), and who fails to engage his audience because the first two equation-filled slides (perfectly acceptable in her department) triggered the audience’s mathophobia.

2. Familiarity breeds comfort. Start with something you are sure most of your audience already knows. Many speakers fear doing that, because, if the audience already knows this stuff, why bother telling them again? Won’t they be bored? Well, if your entire seminar covers what they already know, then the answer would be yes. But if it’s only the first five or 10 minutes, then your audience will love you for placing them on familiar grounds as a staging area for their foray into uncharted (for them) territories.

3. First motivation, then outline. Many speakers start with an outline. Probably not a good idea. The reasons for why you created such an outline would be lost on your audience. Start with a motivation: what is the scientific question you are asking? Why is it important? If you have a brief story that bolsters that motivation then tell it. This could be anything from an interesting clinical case for a health-related seminar to a counter-intuitive or surprising bit of logic for a computational seminar. An initial compelling story with an unresolved problem will create the audience engagement you want. Like any good  campfire storyteller, your goal here is to get your audience to want to hear more. Now that you have hooked them with a motivation and an unresolved story, present your outline.

A laudable goal. You could be more specific though.

4. <cliche>Say what you are going to say, say it, then say what you have just said</cliche>. If you have attended any kind of “improve your public speaking” seminar, this point must have been stated 1,000 times. So for the 1,001st time: yes, do it. Your audience is not familiar with this material as you are. You need to state things several times, without seeming repetitive. This rule can (and should) be used in a nested fashion: apply it to your whole talk, and then apply it to every important sub-section of your talk.

For your entire talk. Say what you are going to say: “Today, I am going to talk about how to survive the zombie apocalypse”.

Say it: (your talk: acquiring food, water, guns, securing the perimeter, etc. etc.)

Then say what you’ve just said: “Therefore, you have to be very determined, have a good aim, travel in groups and don’t make noise at night”.

5. Use new jargon/abbreviations/acronyms/initialisms sparingly. Generally, only use abbreviations if they are (1) common jargon for you and your audience (2) they really are essential to save time. Other than that, avoid them. When introducing new abbreviation/jargon, make sure that you repeat the components and the acronym several times, over several slides. In that way, even your most inattentive audience member should have got it. That being said, do use the jargon common to you and your audience. This will make everyone more comfortable.

6. Explain the graph. When you present a figure with a graph, explain exactly what the X-axis represents, and what the Y-axis represents. Remember: you spent a long time generating this graph and poring over its meaning, but your audience may be seeing it for one minute tops To them it is completely new. Give them time to take it in, and walk them gently through it, holding heir hand. If the graph shows the results of an experiment, make sure to say something like: “if the flies were all expressing the stinky gene, then we should expect the red line to look like this, and the blue dots like that. If  none of the flies were expressing the stinky gene, then the red line would be here, and the blue dots there. BUT BECAUSE (raise voice) the red line is here, and the blue dots are scattered, it means that only this batch of flies expressed Stinky”. (Replace flies with Raccoons where appropriate.)

This may take some time to explain

6. When outlining methods, cartoons work best. When explaining the methods, a flowchart with graphics beats bullet-point sentences every time. ‘Nuff said.

Gene W Yeo, Nicole G Coufal, Tiffany Y Liang, Grace E Peng, Xiang-Dong Fu & Fred H Gage Nature Structural & Molecular Biology 16, 130 - 137 (2009)

7. Spell out the conclusions. Don’t expect your audience to infer the conclusions themselves. It took you weeks to understand your mess of a lab notebook, so how can you expect them to do that in fifty minutes?

8. Be aggressively hesitant. All research talks have that “sketchy bit” in them, where you are not really sure if these results mean anything, or mean what you think they do. Many speakers try to sweep that part under the carpet, or downplay the problem. Don’t do that. On the contrary,  highlight the problem. Everybody has this type of problem in their lab, and you will create and instant bonding by saying: “hey, I’m not sure what is going on here either”. Another consideration is that, if you attempt to hide or downplay your “sketchy bit”, someone from the audience will call you out. That may unravel the good part of your presentation together with the part you tried to hide. Not good.

9. You should have listened to your mom. Poise may be less important in academia than in a business setting. Still, wear a clean shirt, don’t fidget, stand straight, don’t mumble, enunciate and maintain eye contact.  Also, take your finger off the laser pointer trigger unless you are aiming it at the screen. While on the subject, use the pointer sparingly. It’s not a security blanket.

10. Less is more: Don’t read your seminar from slides, that’s pathetic. Your slides are not  the place to put the stuff you are supposed to say, that’s what your gray matter is for, or if you have to, index cards. Words on your slides should be kept to a minimum. Slides are for explanatory figures, graphs, and very brief statements.

11. Be on time. Shoot for five minutes less time than you are allotted. When rehearsing if you are going overtime, there is only one solution. No, speaking faster is not it. No, combining slides is not it either. Killing some of your material is. Yes, that material is precious, and important, and you worked very hard to get these data. It all amounts to a hill of beans if your audience is fidgeting in their chairs 15 minutes after the seminar was supposed to have ended. Newsflash: at this point you are not the brilliant scientist anymore (if you ever were), you are that tiresome nag that is keeping them from lunch, or uncollegially cutting into the next speaker.

12. Rehearse, rehearse rehearse. Boring, yeah. Do it anyway. A lot. Rehearse in front of your cat, your wall, your spouse, your lover, your drinking buddies, anyone you can get a hold of. You will be bored, they will be bored, and you will improve.

One really cool project is run by  Dr. Jim Hu, Dr. Debby Siegele, Dr. Adrienne Zweifel and Dr. Brenley McIntosh . CACAO is an annotation competition and an educational effort. Students around the world form teams which compete to correctly annotate a given set of genes. The gold standard are experimental publications: there are many papers that have the correct experimental data about gene and gene product function, but they are impossible to text-mine automatically. This is where competing teams of students come in: they receive a set of genes, and employ all means to look them up and properly annotate them. The genes the students are given are initially annotated by homology only: each gene was assigned a function based on the similarity of its sequence to another gene that is annotated with a function. IEA or Inferred by Electronic Annotation is the most common method by which annotations are assigned to genes. It is also the least reliable, as it is not scrutinized by any human, and errors may creep in and often do.  The students look for research papers that report experimental evidence for the genes, and correct or validate their annotation.

The scoring system is deliciously cut-throat: after a team posts its own annotations, the other team can look at them. And if team B finds an error in team A’s annotations, team A’s points are transferred to team B.

Credit: (nf) nunoferreira, Flickr

CACAO is primarily an educational effort, teaching students in-depth exploration of scientific literature, and learn how the scientists discover how genes work: because the teams have incentive to be critical of each other’s annotations, the annotations are of a very good quality.

So if you are a life-science teacher with a good group of students who would be interested in this challenge, Hu & gang are looking for CACAO competitors for the spring semester. Also, if you are student, see if you can talk to the faculty in your program about participating in CACAO, and getting some academic credit for it.

More on CACAO, including contact information at the CACAO site.

Also, here is a presentation, given by Prof. Hu at the 2010 GO CAMP.

Muahahaha!