Amelia has over 25 yrs of experience in the IT, Training and Education, and eLearning industry.
The following shortcuts and tips can be applied to most versions of Microsoft Office, up to and including Office 2016. First, I've listed general tips that you can apply to all Microsoft Office applications. Next, I have listed specific tips for Word, Excel, and PowerPoint. Many of them will save you lots of time. I hope you find these tips very helpful!
What kind of tips does this article contain? Here are some of the topics you will find.
Table Formatting and Repeat Header in Word.
Changing text case in Word.
Running a PowerPoint Slideshow from your desktop.
You don't always have to start with a blank Word document, Excel spreadsheet or PowerPoint presentation. There are a variety of pre-formatted templates from you to choose from. Just click File, New (or in 2007, click on the circular Office Button, then click New), then choose a template from the list. Some versions of Office offer Wizards, which guide you through a series of steps to create your customized document.
If you share documents with other co-workers, family or friends, and they are using older versions of Office, you can save your documents in the old Word default (*.doc) vs. new file format (.docx) so your files are backwards compatible. From the File, Save As dialog box, change "Save as type" to .doc vs. docx for Word, xls vs. xlsx for Excel, and ppt vs. pptx for PowerPoint.
To copy formatting in Word, Excel or PowerPoint, including text font, size, color and other attributes, click on the text which contains the formatting you wish to duplicate; then click on the Format Painter icon (cursor will change to a paintbrush), then click in the destination area where you would like to apply formatting. If there are multiple areas you wish to copy formatting, double-click rather than single click the Format Painter icon; this allows you to click in as many areas as you like to apply button; then press ESC when finished.
In Microsoft Office, along with most other applications, you can resize pictures and images by selecting the picture and then using the mouse to drag it to a new size. However, it is important to retain the aspect ratio so your image will not be misshaped. To keep the vertical-to-horizontal ratio (the aspect ratio) the same, hold down the Ctrl key while you drag to the new size. In some applications, you can also use the SHIFT key while you drag to keep the aspect ratio. To crop an image in Microsoft Office, double-click the image, then select Crop from the toolbar.
The Undo command allows you to undo your most recent actions. To Undo the last command, press CTRL+Z. The Redo command lets you change your mind back and restore your original action for those moments when you're really unsure about what you're doing. To Redo a command, press Ctrl+Y. You can also find Undo and Redo either in the Edit menu is older versions of Office, or on the upper left quick access toolbar in Office 2007.
You can enter data in one or more cells and then use the AutoFill handle to extend and fill the selection. Try it with text: type 'January' or 'Jan 01', press ENTER, then point to the bottom right of cell (until you see thin black cross rather than thick cross), then drag down and select area to fill. Excel will try to fill it in for you. For more options, search for AutoFill in Help.
To center text across many columns, such as a title, select the text, then use the Merge and Center icon along the formatting toolbar.
To select a cell and edit the contents, double-click on the cell, or press F2, then edit as needed; press ENTER when finished to accept your changes.
Excel - What Day Is It?
To have the day of the week displayed for one or more dates, first, enter the date(s) in cell(s). Then simply use a custom format to display the day of the week. Select the cell that contains the date(s) or the whole column if needed; then, choose Format, Cells (right click in 2007 to use shortcut menu) and, in the Category list box, select Custom. Now, enter dddd in the Type field, then click OK to return to your spreadsheet. Instead of displaying dates, Excel will display the days of the week which correspond to the dates entered.
To format a worksheet to fit on one piece of paper, select File, Page Setup (in 2007, click Page Layout from the Page Layout tab). In the Page Setup dialog box, select Landscape (assuming your worksheet is wide). Next, in the Scaling section click the Fit To option and then use the arrows to indicate the number of pages wide. Click the Print Preview button to check the results. If the print is too small, go back to Scaling and try the Adjust to option. Try 75% or 85% to manually shrink your page down so it will fit on one page.
To insert a blank sheet into your workbook, just right click on the last sheet tab at the bottom of your worksheet and select Insert New Worksheet. In 2007, there is a new Insert Worksheet tab available that you can click on. In newer versions of Excel, clicl the plus sign in the circle next to your default sheet to insert a new sheet tab.
Need to change the tab name? Just double-click on the sheet tab name and type a new name.
To change the line spacing for a paragraph quickly, position the cursor in the paragraph you would like to modify. Then, press Ctrl+1 to set the spacing at single space, Ctrl+2 to set double spacing, or Ctrl+5 to set the spacing at 1.5 lines. You can modify an entire document or parts of it by first selecting the text you would like to modify and then pressing one of these simple keyboard shortcuts. For more options, right click and select Paragraph, Spacing.
To go to the beginning or end of a line, press HOME or END key.
To select a paragraph, triple in the paragraph.
To select all, click CTRL+A, or in Word, you can triple click in the left margin to select the whole document.
Say you're trying to make a sign that says Wet Paint and you want the text centered, about halfway down the page. The quick and dirty old-school method was to press Enter a dozen or so times until your cursor seemed to be about where you wanted. But haven't you always wanted to begin typing in the middle of a blank page without having to press Enter repeatedly? Beginning with Word 2000, there is a new Click and Type feature which makes this possible. In the new document, simply double-click at any point in the document and begin typing.
To take all text and formatting from one document and insert it into another, you could select and copy from the original document, open the new document, and paste that text in. But there is an easier way. Position the cursor in your new document at the point where you want to insert the old document. Then, in Word, select Insert, Object, Text from File, select the document, and click Insert. In older versions of Word, select Insert, File, select the document; and click Insert.
In Word 2007, select Page Borders from the Page Layout menu (In older versions of Word: select Format, Borders and Shading and click the Page Border tab.) Select the kind of border you would like from the left column, and choose the style of line, color and width from the center column.
In newer versions of Word, click the Borders icon in the Paragraph group on the Home tab in the Ribbon. There's also a drop-down menu next to the Borders icon for more border options.
You can select a fancy border with graphics by clicking on the Art drop down menu.
If you are working in a large document, you may need to be able to view two parts of it simultaneously. The split screen feature allows you to view the document in two separate windows at the same time. To split the screen, select View Split Window (or Window, Split in older versions). Position the this line where you want your document to split and click once. Your single window divides into two windows at this point, each with its own scroll bar. You can now scroll back on the top window to view the earlier portion of the document while viewing your latest edits in the bottom window. To remove the split, simply choose View, Remove Split (or Window, Remove Split), or just drag the line which is splitting your document up to the top of the document.
Have you ever forgotten to capitalize the words in a title? Select the word, sentence, or paragraph you want to modify. Then press SHIFT+F3 as many times as needed to scroll through the available cases, including all caps, initial caps or lower case.
To have the top row appear on every page on table, when in a table, from the Design menu, click the Header Row checkbox. In older versions of Office: first, select the rows you want to appear as a header row. Then, choose Table, Table Properties and click the Row tab. Make sure Repeat As Header Row At Top Of Each Page is checked and click OK. Your selected header row will appear at the top of each page of your table.
You can automatically insert the date and time, in any number of formats. First, select Insert, Date and Time, and you will see a list of formats. Select a date format and click OK. If you check the Update Automatically box, Word will insert a date field and update it automatically.
There's no cents sign on the keyboard, but to create that symbol in Word, press Ctrl+/ (slash) and then press C. To use other symbols, click Insert, Symbol and select from the list.
If you need to keep words together, just as a hyphenated word or a proper name, so that they are not separated on another line, you can use Word's non-breaking space. Type your first word, then press Ctrl+Shift+Spacebar, then type then second word. For example, type: John Ctrl+Shift+Spacebar Smith to keep "John Smith" together. Using the non-breaking space will ensure that John Smith always stay on the same line.
To run a slide show from your desktop, simply save the presentation to your desktop and save it with a PPS file extension. Select File, Save As. Click the Save As Type drop-down arrow and select PowerPoint Show (*.ppsx) from the list. When you double-click a file with a .ppsx extension, which stands for PowerPoint Show, the file opens in Slide Show view. When the slide show finishes, you're returned to the desktop. You can open a .ppsx file for editing by launching PowerPoint, choosing the File menu's Open command, and selecting the .ppsx file.
When you need to put an object on every slide in a slide show, don't waste time trying to make sure all of the objects are perfectly aligned on each slide - use the Slide Master instead. Let's say you want your company logo on the same place on each slide. Open a new slide and choose View, Master, Slide Master. Next, Insert, Picture and locate the file. To get back to your slide show, choose View, Slides (or View, Normal in 2007). At this point, you'll see the logo just where you placed it. You'll also see the logo on any additional slides. If you need to resize or change the logo, return back to you master slide to make the change.
PowerPoint templates offers a variety of templates and themes to choose from. This includes business templates, calendars, flyers, presentations, and even greeting cards.
The newer versions of the PowerPoint offer a bit more templates than the older versions. Select File, New, then select a template from the list. Some are blank with placeholder text boxes for you to fill in, while others have sample content and graphics. The possibilities are endless, so give it a try!
21 months ago from U.S.
I just updated this hub with a link to new video series for Microsoft Word that I'm working on. All the instructional computer videos are free (on youtube). I'm also happy to announce a new computer how-to book series that I'm developing. The first book in the series is available on Amazon. I posted link and info above for the book and video series.
Thanks for letting me know about the other software tool. So far, Scrivener seems pretty easy to use, but I know what you mean. We want to be writing and not figuring out all the software functions. I guess it's a catch 22. I feel like I need Scrivener at this point to help me organize content.
Scriver looks similar to a programme I use call 'Write your own Novel. I find this useful because it has lots of back up in the form of timelines and character information. The only trouble with this sort of thing is that I find it distracting and I spend more time fiddling about instead of working.
thanks for the tip about scrivener. I've not heard of it, but it looks like a good buy.
P.S. When writing, I use Word a lot but recently I started using Scrivener to organize the content from Word. You might like the software too as an author.
nice hub, useful and informative.
I'm an author and use MSword everyday, I've used it since 97 and although it has had a few tweeks in that time, it is still probably the best on the market.
Hi poetryman6969, I think you're right. MS Office, especially Word is so widely used and it's important for job seekers to know all the in's and out's of the software. They say most of us only use a small percentage of all capabilities of a software program. Hopefully my hubs covers some bells and whistles to help others.
Thanks for your comment, glad you found it helpful!
A lot of useful information. I think that young people who are having trouble finding a job need to brush up on skills like these so that they can show employers they have skills that can be useful in an office.
Glad you found my hub very helpful. I appreciate your comments and voting it up and sharing, thanks!
this is a very useful hub indeed. especially, if i wanted to apply for a job or to simply cut the amount of time i spend on a document. voted up and shared.
although I knew a few, I learnt a few as well. Thanks easylearningweb for this useful hub.
Thanks for you comment, anonymous.
Thanks, Birdslover, glad you found the hub helpful!
Its always better to use keyboard than Mouse, it increases the speed, thanks for sharing shortcut keys, I really didn't know some of them.
Was just browsing and came across this article.Very nicely written and easy to understand.
Always glad to helpful, De Greek. I have lots more tips for Word and other MS Office applications, so stay tuned for more hub pages coming soon!
Your welcome, sherrylou57. Glad you found this hub helpful!
Thank you, easyleaningweb for the great tips!
Yep, the good ole autofill..can be very handy! Glad you found it helpful!
Glad it will be a help and thanks for bookmarking the hub!
Aha, so THAT's how you autofill the months in Excel. This tip alone will save me some typing when setting up spreadsheets. Nice!
WoW - nice tips - I bookmarked this one - voted up and useful!
this is very useful in fact I finally now undstand the undo and redo option a lot better. Thanks for this hub, I'll send a few people this way!


When I read yesterday evening a CQ story (subscription required) in which Senator Charles Schumer stated that the Republican filibuster of the Halligan nomination "declares the Gang of 14 agreement null and void" and that "Democrats feel that this really violated the agreement," I had to chuckle at Schumer's brazen shamelessness. There was the architect of the Democrats' unprecedented campaign of filibusters against Bush 43 nominees, and thus a lead target of the Gang of 14 Agreement, having the gall to claim that Republicans were violating an agreement that expired by its own terms five years ago and that Schumer himself never accepted as in any way constraining his own conduct.
The Washington Post cites the Gang of 14 Agreement and notes that the "four GOP members of that group who remain in the Senate" voted against cloture on Halligan. Like the NYT, it doesn't note that the agreement expired long ago. Even worse, it asserts that "Senate Republicans … noted that Schumer and other Democrats had backed judicial filibusters before the Gang of 14 agreement"—and it thus leaves the wildly false impression that Schumer and most other Democrats abandoned the filibuster after the Gang of 14 agreement was reached.
What happened to the Bush-era bipartisan "Gang of 14" and its pledge to reserve filibusters for those "extraordinary circumstances" in which a nominee's qualifications or ethics are suspect or her views far outside the mainstream?
The Los Angeles Times, Roll Call, CNN, NPR, and ABC all provide similar mistreatment. (That's the entire universe of articles I ran across in How Appealing's aggregation.) Not a single article sees fit to note that the Gang of 14 agreement expired five years ago.
Because there were 45 Democrats in the Senate at the time, the apparent commitment by seven Democrats to support cloture (except in "extraordinary circumstances") meant that Democrats would generally no longer be able to filibuster judicial nominees. On the other hand, the Republican signatories to the agreement also committed to oppose the proposed cloture reform, so that effort was dead.
The Gang of 14 agreement expressly related only "to pending and future judicial nominations in the 109th Congress"—that is, for 2005 and 2006. In other words, the agreement by its own terms stopped having any operative effect at the end of 2006.
2. Senator Schumer, then-Senator Obama, and other Democrats who were not signatories to the Gang of 14 agreement never regarded that agreement as somehow limiting their freedom to filibuster judicial nominees. That explains how they voted against cloture on the three pending nominees (Brown, Pryor, and Owen) who were specifically named in the agreement and how Schumer and 32 other Democrats (not including Obama, as it happens) voted against, or failed to support, cloture on the nomination of Brett Kavanaugh to the D.C. Circuit.
3. After the agreement expired at the end of 2006, there is no reason that any signatory (much less any non-signatory) would have regarded it as binding. Thus, in October 2007, one Democratic signatory, Senator Landrieu, voted against cloture on the nomination of Leslie Southwick to the Fifth Circuit—a nomination that no one could plausibly claim presented "exceptional circumstances" (insofar as that term can be thought to have any meaning).
More to come in my Part 2 post.


text:
The Life and Work of Samuel Phelps – Johnston Forbes-Robertson and W. May Phelps
March 21, 2017 March 21, 2017 albanlow	Tags: 1886, Sampson Low Marston Searle and Rivington, Samuel Phelps, Sir Johnston Forbes-Robertson, Theatre, W. May Phelps
Samuel Phelps
Title: The Life and Life-Work of Samuel Phelps:
With Three Portraits and Copies of Letters From Men of Eminence and Other Original Documents of Interest to Play-Goers
Author: W. May Phelps and Sir Johnston Forbes-Robertson
Publisher: Sampson Low, Marston, Searle and Rivington
"As a manager and actor, determined to give of his best to the public, and put into his work his whole heart and soul and strength, Samuel Phelps was without fear; and, as a man, without reproach. In every relation of life he was tender, dignified, and righteous ; and in what pertained to the traditions and glory of the English stage, he was the last of all the Romans."
Samuel Phelps (1804 – 1878) was an English actor and theatre manager. He is known for his productions of William Shakespeare's plays which were faithful to their original versions. Phelps made his début as Shylock in London at the Haymarket Theatre in 1837. He took over the management of the then-unfashionable Sadler's Wells Theatre and revolutionised the production of Shakespeare's plays by restoring Shakespearean performances to the original text of the first folio and away from the adaptations by Colley Cibber, Nahum Tate and David Garrick that had been favoured by the theatre-going public since the Restoration. Phelps staged all but four of Shakespeare's plays at Sadler's Wells, some of which (like The Winter's Tale and Measure for Measure) hadn't been performed since their premieres at the Globe Theatre.
Phelps' most frequently performed role was Hamlet, but he counted Macbeth, Wolsey and Bottom among his greatest achievements. He was generally considered the finest King Lear of his generation, returning to Shakespeare's version. Phelps other great creation was his Fastaff, which the German publication Gesammelte Werke called his finest role. Sadly, Phelps' skills declined in old age so that critics no longer cared for his work in tragic plays, approving only his performances in comic roles like Falstaff and Bottom. But in his prime, he was the most versatile actor of his generation.
Sir Johnston Forbes-Robertson
Sir Johnston Forbes-Robertson (1853 – 1937) was an English actor and theatre manager. He was considered the finest Hamlet of the Victorian era and one of the finest actors of his time. Born in London, he was the eldest of the eleven children of John Forbes-Robertson, a theatre critic and journalist from Aberdeen, and his wife Frances. He was educated at Charterhouse. Originally intending to become an artist, he trained for three years at the Royal Academy. He began a theatrical career, out of a desire to be self-supporting, when the dramatist William Gorman Wills, who had seen him in private theatricals, offered him a role in his play Mary Queen of Scots. His many performances led him into, among other things, travel to the U.S., and work with Sir Henry Irving. He was hailed as one of the most individual and refined of English actors.
A statue of Forbes-Robertson by Brenda Putnam (1932) can be found at the Folger Shakespeare Library in Washington D.C. and there is a memorial plaque in Bedford Square, London.
W. May Phelps was the nephew of Samuel Phelps.
This book is twinned with Theatre of Rags by Francesca Albini (pub. 2017) read more about it here.
← Breathe Through This If You Can – Kevin Acott
Connection – Dimitra Petsa, Charlotte Bell and Rosie Rosenberg →

redpajama_set_name: RedPajamaCommonCrawl


text:
Solar Directory
Solar Evaluation
Bookmarked Feeds
Rated Company
Select Location Alabama Alaska Arizona Arkansas California Colorado Connecticut Delaware District Of Cloumbia Florida Georgia Hawaii Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Maryland Massachusetts Michigan Minnesota Mississippi Missouri Montana Nebraska Nevada New Hampshire New Jersey New Mexico New York North Carolina North Dakota Ohio Oklahoma Oregon Pennsylvania Puerto Rico Rhode Island South Carolina South Dakota Tennessee Texas Utah Vermont Virginia Washington West Virginia Wisconsin Wyoming
Select Employee above 10000 5001 to 10000 1001-5000 501-1000 201-500 51-200 11-50 1-10
Select Revenue more than $10,000 million $1,000 to $10,000 million $5,00 to $1000 million $50 to $500 million $10 to $50 million $5 to $10 million $1 to $5 million Less than $1 million
Select Likes more than 100K 50K to 100K 25K to 50K 10K to 25K 5K to 10K 1K to 5K Less than 1K
Select monthly visitors more than 100K 50K to 100K 25K to 50K 10K to 25K 5K to 10K 1K to 5K Less than 1K
Please Sign-in
By Revenue
By Employee
Solar Evaluation Companies in United States (1)
Mesiti-Miller Engineering, Inc Civil and structural engineering design services for Solar Mounting Systems
Mesiti-Miller Engineering was established in 1987 and incorporated in 1994 to provide professional civil and structural engineering services to both ...more about Mesiti-Miller Engineering, Inc
Most Searched Company
VivoPower International
First Solar, Inc.
SunPower Corporation
Strata Solar, LLC
Vivint Solar, Inc


Welcome to Virtual Innovation Space!
The below terms are effective April 1, 2014.
Thanks for using Virtual Innovation Space ("VIS"). By using VIS, you are agreeing to these terms. Please read them carefully.
You must follow any policies made available to you within the VIS.
Don't misuse VIS. For example, don't interfere with VIS or try to access it using a method other than the interface and the instructions that we provide. You may use VIS only as permitted by law, including applicable export and re-export control laws and regulations. We may suspend or stop providing VIS to you if you do not comply with our terms or policies or if we are investigating suspected misconduct.
Using VIS does not give you ownership of any intellectual property rights in VIS or the content you access. You may not use content from VIS unless you obtain permission from its owner or are otherwise permitted by law. These terms do not grant you the right to use any branding or logos used in VIS. Don't remove, obscure, or alter any legal notices displayed in or along with our Services.
VIS displays some content that is not ours. This content is the sole responsibility of the entity that makes it available. We may review content to determine whether it is illegal or violates our policies, and we may remove or refuse to display content that we reasonably believe violates our policies or the law. But that does not necessarily mean that we review content, so please don't assume that we do.
In connection with your use of the VIS, we may send you service announcements, administrative messages, and other information. You may opt out of some of those communications.
VIS privacy policies explain how we treat your personal data and protect your privacy when you use our system. By using VIS, you agree that we can use such data in accordance with our privacy policies. We respond to notices of alleged copyright infringement and terminate accounts of repeat infringers.
VIS allows you to upload, submit, store, send or receive content. You retain ownership of any intellectual property rights that you hold in that content. In short, what belongs to you stays yours.
When you upload, submit, store, send or receive content to or through VIS, you give us (and those we work with) a worldwide license to use, host, store, reproduce, modify, create derivative works (such as those resulting from translations, adaptations or other changes we make so that your content works better with VIS), communicate, publish, publicly perform, publicly display and distribute such content. The rights you grant in this license are for the limited purpose of operating, promoting, and improving VIS, and to develop new services. This license continues even if you stop using VIS. Make sure you have the necessary rights to grant us this license for any content that you submit to VIS.
If you have a VIS Account, we may display your Profile name, Profile photo, and actions you take on VIS or on third-party applications connected to your VIS Account (such as reviews you write and comments you post) in VIS. We will respect the choices you make to limit sharing or visibility settings in your VIS Account. For example, you can allow other users to contact you via a personal contact form which keeps your e-mail address hidden.
If you submit feedback or suggestions about VIS, we may use your feedback or suggestions without obligation to you.
OTHER THAN AS EXPRESSLY SET OUT IN THESE TERMS OR ADDITIONAL TERMS, NEITHER VIS OPERATOR NOR TEMPUS NEFESIE CONSORTIUM MAKE ANY SPECIFIC PROMISES ABOUT THE SERVICES. FOR EXAMPLE, WE DON'T MAKE ANY COMMITMENTS ABOUT THE CONTENT WITHIN THE SERVICES, THE SPECIFIC FUNCTIONS OF THE SERVICES, OR THEIR RELIABILITY, AVAILABILITY, OR ABILITY TO MEET YOUR NEEDS. WE PROVIDE THE SERVICES "AS IS".
WHEN PERMITTED BY LAW, VIS OWNERS AND/OR OPERATORs, WILL NOT BE RESPONSIBLE FOR LOST PROFITS, REVENUES, OR DATA, FINANCIAL LOSSES OR INDIRECT, SPECIAL, CONSEQUENTIAL, EXEMPLARY, OR PUNITIVE DAMAGES.
TO THE EXTENT PERMITTED BY LAW, THE TOTAL LIABILITY OF VIS OWNERS, OPERATORS, AND TEMPUS NEFESIE CONSORTIUM, FOR ANY CLAIMS UNDER THESE TERMS, INCLUDING FOR ANY IMPLIED WARRANTIES, IS LIMITED TO THE AMOUNT YOU PAID US TO USE THE SERVICES (OR, IF WE CHOOSE, TO SUPPLYING YOU THE SERVICES AGAIN).
IN ALL CASES VIS OWNERS AND/OR OPERATORS WILL NOT BE LIABLE FOR ANY LOSS OR DAMAGE THAT IS NOT REASONABLY FORESEEABLE.
If you are using VIS on behalf of a business, that business accepts these terms. It will hold harmless and indemnify VIS and its owners, operators, and users from any claim, suit or action arising from or related to the use of the VIS or violation of these terms, including any liability or expense arising from claims, losses, damages, suits, judgments, litigation costs and attorneys' fees.
These terms control the relationship between VIS and you. They do not create any third party beneficiary rights.


text:
Jodi Proznick
Triology Album
Formed in 2008, Triology is a Jazz Trio featuring three of Canada's most sought after jazz musicians: Jodi Proznick (bass), Bill Coon (guitar), Miles Black (piano). This group was fashioned after the great early jazz trios of Nat King Cole and Oscar Peterson, and features wonderful arrangements of classic standards and beautiful original tunes by the trio members.
1 review for Triology Album
Mary K – October 1, 2017
If you really pressed me about it, I'd probably cast my vote for the Most Swinging Piano Award as a three way tie between Art Tatum, Nat Cole and Oscar Peterson. They all led successful trios of piano, bass and guitar. And this frolicking, infectious threesome seems to have roots in that musical garden. They even open their program with "Ray Time," a tribute to Ray Brown, Oscar's world-renowned bassist; and pianist Miles Black has composed a very "Ray-like" opener. The standards on the album include an infectious "Sweet Georgia Brown"; a version of "If I Love Again," taken at a slower tempo than the one you remember from Clifford Brown; some sprightly communication on "Pennies From Heaven"; and the closer, a swift "I Got Rhythm." Among several originals by various group members, I especially liked the fresh, happy flavor of "Broadway and Alma," and the confident, forward-moving tempo of "Adanac" (try spelling it backwards). And before I forget, these three swinging players are Bill Coon, guitar, Jodi Proznick, bass, and Miles Black, piano. They are continuing an honored piano tradition, and kudos to them for doing it so well. I hope to hear more. – George Fendel, The Jazz Society of Oregon
Sun Songs Album
Foundations – Digital Download
Triology: Stairway to the Stars
Designed by Learnbase


text:
package org.opencb.opencga.storage.hadoop.variant.index.sample;

import com.google.common.collect.Iterators;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.io.compress.Compression;
import org.apache.hadoop.hbase.util.Bytes;
import org.opencb.biodata.models.core.Region;
import org.opencb.biodata.models.variant.Variant;
import org.opencb.commons.datastore.core.ObjectMap;
import org.opencb.commons.datastore.core.Query;
import org.opencb.commons.datastore.core.QueryOptions;
import org.opencb.opencga.storage.core.exceptions.StorageEngineException;
import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager;
import org.opencb.opencga.storage.core.metadata.models.SampleMetadata;
import org.opencb.opencga.storage.core.metadata.models.StudyMetadata;
import org.opencb.opencga.storage.core.metadata.models.TaskMetadata;
import org.opencb.opencga.storage.core.utils.iterators.CloseableIterator;
import org.opencb.opencga.storage.core.utils.iterators.IntersectMultiKeyIterator;
import org.opencb.opencga.storage.core.utils.iterators.UnionMultiKeyIterator;
import org.opencb.opencga.storage.core.variant.VariantStorageOptions;
import org.opencb.opencga.storage.core.variant.adaptors.VariantIterable;
import org.opencb.opencga.storage.core.variant.adaptors.VariantQueryException;
import org.opencb.opencga.storage.core.variant.adaptors.VariantQueryParam;
import org.opencb.opencga.storage.core.variant.adaptors.iterators.IntersectMultiVariantKeyIterator;
import org.opencb.opencga.storage.core.variant.adaptors.iterators.UnionMultiVariantKeyIterator;
import org.opencb.opencga.storage.core.variant.adaptors.iterators.VariantDBIterator;
import org.opencb.opencga.storage.core.variant.query.VariantQueryUtils;
import org.opencb.opencga.storage.core.variant.query.VariantQueryUtils.QueryOperation;
import org.opencb.opencga.storage.hadoop.utils.HBaseManager;
import org.opencb.opencga.storage.hadoop.variant.GenomeHelper;
import org.opencb.opencga.storage.hadoop.variant.HadoopVariantStorageOptions;
import org.opencb.opencga.storage.hadoop.variant.index.IndexUtils;
import org.opencb.opencga.storage.hadoop.variant.index.core.filters.IndexFieldFilter;
import org.opencb.opencga.storage.hadoop.variant.index.query.*;
import org.opencb.opencga.storage.hadoop.variant.utils.HBaseVariantTableNameGenerator;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;
import java.io.UncheckedIOException;
import java.util.*;
import java.util.stream.Collectors;

import static org.opencb.opencga.storage.hadoop.variant.adaptors.phoenix.VariantSqlQueryParser.DEFAULT_LOADED_GENOTYPES;
import static org.opencb.opencga.storage.hadoop.variant.index.IndexUtils.EMPTY_MASK;

/**
 * Created on 14/05/18.
 *
 * @author Jacobo Coll <jacobo167@gmail.com>
 */
public class SampleIndexDBAdaptor implements VariantIterable {

    private final HBaseManager hBaseManager;
    private final HBaseVariantTableNameGenerator tableNameGenerator;
    private final VariantStorageMetadataManager metadataManager;
    private final SampleIndexSchemaFactory schemaFactory;
    private static byte[] family = GenomeHelper.COLUMN_FAMILY_BYTES;
    private static Logger logger = LoggerFactory.getLogger(SampleIndexDBAdaptor.class);

    public SampleIndexDBAdaptor(HBaseManager hBaseManager, HBaseVariantTableNameGenerator tableNameGenerator,
                                VariantStorageMetadataManager metadataManager) {
        this.hBaseManager = hBaseManager;
        this.tableNameGenerator = tableNameGenerator;
        this.metadataManager = metadataManager;
        this.schemaFactory = new SampleIndexSchemaFactory(metadataManager);
    }

    @Override
    public VariantDBIterator iterator(Query query, QueryOptions options) {
        return iterator(parseSampleIndexQuery(query));
    }

    public VariantDBIterator iterator(SampleIndexQuery query) {
        return iterator(query, QueryOptions.empty());
    }

    public VariantDBIterator iterator(SampleIndexQuery query, QueryOptions options) {
        Map<String, List<String>> samples = query.getSamplesMap();

        if (samples.isEmpty()) {
            throw new VariantQueryException("At least one sample expected to query SampleIndex!");
        }
        SampleIndexSchema schema = query.getSchema();
        QueryOperation operation = query.getQueryOperation();

        if (samples.size() == 1) {
            String sample = samples.entrySet().iterator().next().getKey();
            List<String> gts = query.getSamplesMap().get(sample);

            if (gts.isEmpty()) {
                // If empty, should find none. Return empty iterator
                return VariantDBIterator.emptyIterator();
            } else {
                logger.info("Single sample indexes iterator : " + sample);
                SingleSampleIndexVariantDBIterator iterator = internalIterator(query.forSample(sample, gts), schema);
                return iterator.localLimitSkip(options);
            }
        }

        List<VariantDBIterator> iterators = new ArrayList<>(samples.size());
        List<VariantDBIterator> negatedIterators = new ArrayList<>(samples.size());

        for (Map.Entry<String, List<String>> entry : samples.entrySet()) {
            String sample = entry.getKey();
            List<String> gts = entry.getValue();

            if (query.isNegated(sample)) {
                if (!gts.isEmpty()) {
                    negatedIterators.add(internalIterator(query.forSample(sample, gts), schema));
                }
                // Skip if GTs to query is empty!
                // Otherwise, it will return ALL genotypes instead of none
            } else {
                if (gts.isEmpty()) {
                    // If empty, should find none. Add empty iterator for this sample
                    iterators.add(VariantDBIterator.emptyIterator());
                } else {
                    iterators.add(internalIterator(query.forSample(sample, gts), schema));
                }
            }
        }
        VariantDBIterator iterator;
        if (operation.equals(QueryOperation.OR)) {
            logger.info("Union of " + iterators.size() + " sample indexes");
            iterator = new UnionMultiVariantKeyIterator(iterators);
        } else {
            logger.info("Intersection of " + iterators.size() + " sample indexes plus " + negatedIterators.size() + " negated indexes");
            iterator = new IntersectMultiVariantKeyIterator(iterators, negatedIterators);
        }

        return iterator.localLimitSkip(options);
    }

    /**
     * Partially processed iterator. Internal usage only.
     *
     * @param query SingleSampleIndexQuery
     * @param schema Sample index schema
     * @return SingleSampleIndexVariantDBIterator
     */
    private SingleSampleIndexVariantDBIterator internalIterator(SingleSampleIndexQuery query, SampleIndexSchema schema) {
        String tableName = getSampleIndexTableName(query);

        try {
            return hBaseManager.act(tableName, table -> {
                return new SingleSampleIndexVariantDBIterator(table, query, schema, this);
            });
        } catch (IOException e) {
            throw VariantQueryException.internalException(e);
        }
    }

    private RawSingleSampleIndexVariantDBIterator rawInternalIterator(SingleSampleIndexQuery query, SampleIndexSchema schema) {
        String tableName = getSampleIndexTableName(query);

        try {
            return hBaseManager.act(tableName, table -> {
                return new RawSingleSampleIndexVariantDBIterator(table, query, schema, this);
            });
        } catch (IOException e) {
            throw VariantQueryException.internalException(e);
        }
    }

    public String getSampleIndexTableName(SampleIndexQuery query) {
        return tableNameGenerator.getSampleIndexTableName(toStudyId(query.getStudy()), query.getSchema().getVersion());
    }

    public String getSampleIndexTableNameLatest(int studyId) {
        int version = schemaFactory.getSampleIndexConfigurationLatest(studyId, true).getVersion();
        return tableNameGenerator.getSampleIndexTableName(studyId, version);
    }

    public String getSampleIndexTableName(int studyId, int version) {
        return tableNameGenerator.getSampleIndexTableName(studyId, version);
    }

    public HBaseVariantTableNameGenerator getTableNameGenerator() {
        return tableNameGenerator;
    }

    public VariantStorageMetadataManager getMetadataManager() {
        return metadataManager;
    }

    protected Map<String, List<Variant>> queryByGt(int study, int sample, String chromosome, int position)
            throws IOException {
        SampleIndexSchema schema = schemaFactory.getSchema(study, sample, false);
        Result result = queryByGtInternal(study, sample, chromosome, position, schema.getVersion());
        return newConverter(schema).convertToMap(result);
    }

    protected SampleIndexEntryPutBuilder queryByGtBuilder(int study, int sample, String chromosome, int position, SampleIndexSchema schema)
            throws IOException {
        Result result = queryByGtInternal(study, sample, chromosome, position, schema.getVersion());
        return new SampleIndexEntryPutBuilder(sample, chromosome, position, schema,
                newConverter(schema).convertToMapSampleVariantIndex(result));
    }

    private Result queryByGtInternal(int study, int sample, String chromosome, int position, int version) throws IOException {
        String tableName = getSampleIndexTableName(study, version);
        return hBaseManager.act(tableName, table -> {
            Get get = new Get(SampleIndexSchema.toRowKey(sample, chromosome, position));
            get.addFamily(family);
            return table.get(get);
        });
    }

    public Iterator<Map<String, List<Variant>>> iteratorByGt(int study, int sample) throws IOException {
        SampleIndexSchema schema = schemaFactory.getSchema(study, sample, false);
        String tableName = getSampleIndexTableName(study, schema.getVersion());
        HBaseToSampleIndexConverter converter = newConverter(schema);
        return hBaseManager.act(tableName, table -> {
            Scan scan = new Scan();
            scan.setRowPrefixFilter(SampleIndexSchema.toRowKey(sample));
            try {
                ResultScanner scanner = table.getScanner(scan);
                Iterator<Result> resultIterator = scanner.iterator();
                return Iterators.transform(resultIterator, converter::convertToMap);
            } catch (IOException e) {
                throw VariantQueryException.internalException(e);
            }
        });
    }

    public CloseableIterator<SampleIndexEntry> rawIterator(int study, int sample) throws IOException {
        return rawIterator(study, sample, null);
    }

    public CloseableIterator<SampleIndexEntry> rawIterator(int study, int sample, Region region) throws IOException {
        SampleIndexSchema schema = schemaFactory.getSchema(study, sample, false);
        String tableName = getSampleIndexTableName(study, schema.getVersion());
        return hBaseManager.act(tableName, table -> {
            Scan scan = new Scan();
            if (region != null) {
                scan.setStartRow(SampleIndexSchema.toRowKey(sample, region.getChromosome(), region.getStart()));
                scan.setStopRow(SampleIndexSchema.toRowKey(sample, region.getChromosome(),
                        region.getEnd() + (region.getEnd() == Integer.MAX_VALUE ? 0 : SampleIndexSchema.BATCH_SIZE)));
            } else {
                scan.setRowPrefixFilter(SampleIndexSchema.toRowKey(sample));
            }

            HBaseToSampleIndexConverter converter = newConverter(schema);
            ResultScanner scanner = table.getScanner(scan);
            Iterator<Result> resultIterator = scanner.iterator();
            return CloseableIterator.wrap(Iterators.transform(resultIterator, converter::convert), scanner);
        });
    }

    /**
     * Partially processed iterator. Internal usage only.
     *
     * @param study study
     * @param sample sample
     * @return SingleSampleIndexVariantDBIterator
     */
    public RawSingleSampleIndexVariantDBIterator rawIterator(String study, String sample) {
        Query query = new Query(VariantQueryParam.STUDY.key(), study).append(VariantQueryParam.SAMPLE.key(), sample);
        SingleSampleIndexQuery sampleIndexQuery = parseSampleIndexQuery(query).forSample(sample);
        return rawInternalIterator(sampleIndexQuery, sampleIndexQuery.getSchema());
    }

    public CloseableIterator<SampleVariantIndexEntry> rawIterator(Query query) throws IOException {
        return rawIterator(parseSampleIndexQuery(query));
    }

    public CloseableIterator<SampleVariantIndexEntry> rawIterator(SampleIndexQuery query) throws IOException {
        return rawIterator(query, new QueryOptions());
    }

    public CloseableIterator<SampleVariantIndexEntry> rawIterator(SampleIndexQuery query, QueryOptions options) throws IOException {
        Map<String, List<String>> samples = query.getSamplesMap();

        if (samples.isEmpty()) {
            throw new VariantQueryException("At least one sample expected to query SampleIndex!");
        }
        SampleIndexSchema schema = query.getSchema();
        QueryOperation operation = query.getQueryOperation();

        if (samples.size() == 1) {
            String sample = samples.entrySet().iterator().next().getKey();
            List<String> gts = query.getSamplesMap().get(sample);

            if (gts.isEmpty()) {
                // If empty, should find none. Return empty iterator
                return CloseableIterator.emptyIterator();
            } else {
                logger.info("Single sample indexes iterator");
                RawSingleSampleIndexVariantDBIterator iterator = rawInternalIterator(query.forSample(sample, gts), schema);
                return iterator.localLimitSkip(options);
            }
        }

        List<RawSingleSampleIndexVariantDBIterator> iterators = new ArrayList<>(samples.size());
        List<RawSingleSampleIndexVariantDBIterator> negatedIterators = new ArrayList<>(samples.size());

        for (Map.Entry<String, List<String>> entry : samples.entrySet()) {
            String sample = entry.getKey();
            List<String> gts = entry.getValue();

            if (query.isNegated(sample)) {
                if (!gts.isEmpty()) {
                    negatedIterators.add(rawInternalIterator(query.forSample(sample, gts), schema));
                }
                // Skip if GTs to query is empty!
                // Otherwise, it will return ALL genotypes instead of none
            } else {
                if (gts.isEmpty()) {
                    // If empty, should find none. Add empty iterator for this sample
                    iterators.add(RawSingleSampleIndexVariantDBIterator.emptyIterator());
                } else {
                    iterators.add(rawInternalIterator(query.forSample(sample, gts), schema));
                }
            }
        }

        final CloseableIterator<SampleVariantIndexEntry> iterator;
        if (operation.equals(QueryOperation.OR)) {
            logger.info("Union of " + iterators.size() + " sample indexes");
            iterator = new UnionMultiKeyIterator<>(
                    Comparator.comparing(SampleVariantIndexEntry::getVariant, VariantDBIterator.VARIANT_COMPARATOR), iterators);
        } else {
            logger.info("Intersection of " + iterators.size() + " sample indexes plus " + negatedIterators.size() + " negated indexes");
            iterator = new IntersectMultiKeyIterator<>(
                    Comparator.comparing(SampleVariantIndexEntry::getVariant, VariantDBIterator.VARIANT_COMPARATOR),
                    iterators, negatedIterators);
        }

        return iterator.localLimitSkip(options);
    }

    public boolean isFastCount(SampleIndexQuery query) {
        return query.getSamplesMap().size() == 1 && query.emptyAnnotationIndex() && query.emptyFileIndex();
    }

    public long count(List<Region> regions, String study, String sample, List<String> gts) {
        return count(newParser().parse(regions, study, sample, gts));
    }

    public long count(SampleIndexQuery query) {
        if (query.getSamplesMap().size() == 1 && query.getMendelianErrorSet().isEmpty()) {
            String sample = query.getSamplesMap().keySet().iterator().next();
            return count(query.forSample(sample));
        } else {
            return Iterators.size(iterator(query));
        }
    }

    private long count(SingleSampleIndexQuery query) {
        Collection<LocusQuery> locusQueries;
        if (CollectionUtils.isEmpty(query.getLocusQueries())) {
            // If no locus are defined, get a list of one null element to initialize the stream.
            locusQueries = Collections.singletonList(null);
        } else {
            locusQueries = query.getLocusQueries();
        }

        int studyId = toStudyId(query.getStudy());
        String tableName = getSampleIndexTableName(studyId, query.getSchema().getVersion());

        HBaseToSampleIndexConverter converter = newConverter(query.getSchema());
        try {
            return hBaseManager.act(tableName, table -> {
                long count = 0;
                for (LocusQuery locusQuery : locusQueries) {
                    // Split region in countable regions
                    List<LocusQuery> subLocusQueries;
                    if (locusQuery != null && locusQuery.getVariants().isEmpty() && locusQuery.getRegions().size() == 1) {
                        subLocusQueries = splitRegion(locusQuery.getRegions().get(0))
                                .stream().map(LocusQuery::buildLocusQuery).collect(Collectors.toList());
                    } else {
                        // Do not split
                        subLocusQueries = Collections.singletonList(locusQuery);
                    }
                    for (LocusQuery subLocusQuery : subLocusQueries) {
                        boolean noLocusFilter = subLocusQuery == null
                                || (subLocusQuery.getVariants().isEmpty()
                                && subLocusQuery.getRegions().size() == 1
                                && matchesWithBatch(subLocusQuery.getRegions().get(0)));
                        // Don't need to parse the variant to filter
                        boolean simpleCount = !query.isMultiFileSample()
                                && CollectionUtils.isEmpty(query.getVariantTypes())
                                && noLocusFilter;
                        try {
                            if (query.emptyOrRegionFilter() && simpleCount) {
                                // Directly sum counters
                                Scan scan = parseCount(query, subLocusQuery);
                                ResultScanner scanner = table.getScanner(scan);
                                Result result = scanner.next();
                                while (result != null) {
                                    count += converter.convertToCount(result);
                                    result = scanner.next();
                                }
                            } else {
                                SampleIndexEntryFilter filter = buildSampleIndexEntryFilter(query, subLocusQuery);
                                Scan scan;
                                if (simpleCount) {
                                    // Fast filter and count. Don't need to parse the variant to filter
                                    scan = parseCountAndFilter(query, subLocusQuery);
                                } else {
                                    // Need to parse the variant to finish filtering. Create a normal scan query.
                                    scan = parse(query, subLocusQuery);
                                }
                                ResultScanner scanner = table.getScanner(scan);
                                Result result = scanner.next();
                                while (result != null) {
                                    SampleIndexEntry sampleIndexEntry = converter.convert(result);
                                    count += filter.filterAndCount(sampleIndexEntry);
                                    result = scanner.next();
                                }
                            }
                        } catch (IOException e) {
                            throw VariantQueryException.internalException(e);
                        }
                    }
                }
                return count;
            });
        } catch (IOException e) {
            throw VariantQueryException.internalException(e);
        }
    }

    protected HBaseToSampleIndexConverter newConverter(SampleIndexSchema schema) {
        return new HBaseToSampleIndexConverter(schema);
    }

    public SampleIndexSchemaFactory getSchemaFactory() {
        return schemaFactory;
    }

    public SampleIndexSchema getSchemaLatest(Object study) {
        return schemaFactory.getSchemaLatest(toStudyId(study));
    }

    protected int toStudyId(Object study) {
        int studyId;
        if (study == null || study instanceof String && ((String) study).isEmpty()) {
            Map<String, Integer> studies = metadataManager.getStudies(null);
            if (studies.size() == 1) {
                studyId = studies.values().iterator().next();
            } else {
                throw VariantQueryException.missingStudy(studies.keySet());
            }
        } else {
            studyId = metadataManager.getStudyId(study);
        }
        return studyId;
    }

    protected List<String> getAllLoadedGenotypes(int study) {
        List<String> allGts = metadataManager.getStudyMetadata(study)
                .getAttributes()
                .getAsStringList(VariantStorageOptions.LOADED_GENOTYPES.key());
        if (allGts == null || allGts.isEmpty()) {
            allGts = DEFAULT_LOADED_GENOTYPES;
        }
        return allGts;
    }

    /**
     * Split region into regions that match with batches at SampleIndexTable.
     *
     * @param region Region to split
     * @return List of regions.
     */
    protected static List<Region> splitRegion(Region region) {
        List<Region> regions;
        if (region.getEnd() - region.getStart() < SampleIndexSchema.BATCH_SIZE) {
            // Less than one batch. Do not split region
            regions = Collections.singletonList(region);
        } else if (region.getStart() / SampleIndexSchema.BATCH_SIZE + 1 == region.getEnd() / SampleIndexSchema.BATCH_SIZE
                && !startsAtBatch(region)
                && !endsAtBatch(region)) {
            // Consecutive partial batches. Do not split region
            regions = Collections.singletonList(region);
        } else {
            // Copy region before modifying
            region = new Region(region.getChromosome(), region.getStart(), region.getEnd());
            regions = new ArrayList<>(3);
            if (!startsAtBatch(region)) {
                int splitPoint = region.getStart() - region.getStart() % SampleIndexSchema.BATCH_SIZE + SampleIndexSchema.BATCH_SIZE;
                regions.add(new Region(region.getChromosome(), region.getStart(), splitPoint - 1));
                region.setStart(splitPoint);
            }
            regions.add(region);
            if (!endsAtBatch(region)) {
                int splitPoint = region.getEnd() - region.getEnd() % SampleIndexSchema.BATCH_SIZE;
                regions.add(new Region(region.getChromosome(), splitPoint, region.getEnd()));
                region.setEnd(splitPoint - 1);
            }
        }
        return regions;
    }

    protected static boolean matchesWithBatch(Region region) {
        return region == null || startsAtBatch(region) && endsAtBatch(region);
    }

    protected static boolean startsAtBatch(Region region) {
        return region.getStart() % SampleIndexSchema.BATCH_SIZE == 0;
    }

    protected static boolean endsAtBatch(Region region) {
        return region.getEnd() == Integer.MAX_VALUE || (region.getEnd() + 1) % SampleIndexSchema.BATCH_SIZE == 0;
    }

    public SampleIndexEntryFilter buildSampleIndexEntryFilter(SingleSampleIndexQuery query, LocusQuery locusQuery) {
        if (locusQuery == null
                || (locusQuery.getVariants().isEmpty() && locusQuery.getRegions().size() == 1
                && matchesWithBatch(locusQuery.getRegions().get(0)))) {
            return new SampleIndexEntryFilter(query, null);
        } else {
            return new SampleIndexEntryFilter(query, locusQuery);
        }
    }

    public SampleIndexQuery parseSampleIndexQuery(Query query) {
        return newParser().parse(query);
    }

    protected SampleIndexQueryParser newParser() {
        return new SampleIndexQueryParser(metadataManager, schemaFactory);
    }

    public Scan parse(SingleSampleIndexQuery query, LocusQuery regions) {
        return parse(query, regions, false, false);
    }

    public Scan parseIncludeAll(SingleSampleIndexQuery query, LocusQuery region) {
        return parse(query, region, false, false, true);
    }

    public Scan parseCount(SingleSampleIndexQuery query, LocusQuery regions) {
        return parse(query, regions, true, true);
    }

    public Scan parseCountAndFilter(SingleSampleIndexQuery query, LocusQuery regions) {
        return parse(query, regions, false, true);
    }

    private Scan parse(SingleSampleIndexQuery query, LocusQuery regions, boolean onlyCount, boolean skipGtColumn) {
        return parse(query, regions, onlyCount, skipGtColumn, false);
    }

    private Scan parse(SingleSampleIndexQuery query, LocusQuery locusQuery, boolean onlyCount, boolean skipGtColumn, boolean includeAll) {

        Scan scan = new Scan();
        int studyId = toStudyId(query.getStudy());
        int sampleId = toSampleId(studyId, query.getSample());
        if (locusQuery != null) {
            // Regions from the same group are sorted and do not overlap.
            Region region = locusQuery.getChunkRegion();
            scan.setStartRow(SampleIndexSchema.toRowKey(sampleId, region.getChromosome(), region.getStart()));
            scan.setStopRow(SampleIndexSchema.toRowKey(sampleId, region.getChromosome(), region.getEnd()));
        } else {
            scan.setStartRow(SampleIndexSchema.toRowKey(sampleId));
            scan.setStopRow(SampleIndexSchema.toRowKey(sampleId + 1));
        }
        // If genotypes are not defined, return ALL columns
        for (String gt : query.getGenotypes()) {
            scan.addColumn(family, SampleIndexSchema.toGenotypeDiscrepanciesCountColumn());
            scan.addColumn(family, SampleIndexSchema.toGenotypeCountColumn(gt));
            if (!onlyCount) {
                if (query.getMendelianError()) {
                    scan.addColumn(family, SampleIndexSchema.toMendelianErrorColumn());
                } else {
                    if (!skipGtColumn) {
                        scan.addColumn(family, SampleIndexSchema.toGenotypeColumn(gt));
                    }
                }
                if (includeAll || query.getAnnotationIndexMask() != EMPTY_MASK) {
                    scan.addColumn(family, SampleIndexSchema.toAnnotationIndexColumn(gt));
                    scan.addColumn(family, SampleIndexSchema.toAnnotationIndexCountColumn(gt));
                }
                if (includeAll || !query.getAnnotationIndexQuery().getCtBtTfFilter().isNoOp()) {
                    scan.addColumn(family, SampleIndexSchema.toAnnotationCtBtTfIndexColumn(gt));
                    // All independent indexes are required to parse CtBtTf
                    scan.addColumn(family, SampleIndexSchema.toAnnotationBiotypeIndexColumn(gt));
                    scan.addColumn(family, SampleIndexSchema.toAnnotationConsequenceTypeIndexColumn(gt));
                    scan.addColumn(family, SampleIndexSchema.toAnnotationTranscriptFlagIndexColumn(gt));
                }
                if (includeAll || !query.getAnnotationIndexQuery().getBiotypeFilter().isNoOp()) {
                    scan.addColumn(family, SampleIndexSchema.toAnnotationBiotypeIndexColumn(gt));
                }
                if (includeAll || !query.getAnnotationIndexQuery().getConsequenceTypeFilter().isNoOp()) {
                    scan.addColumn(family, SampleIndexSchema.toAnnotationConsequenceTypeIndexColumn(gt));
                }
                if (includeAll || !query.getAnnotationIndexQuery().getTranscriptFlagFilter().isNoOp()) {
                    scan.addColumn(family, SampleIndexSchema.toAnnotationTranscriptFlagIndexColumn(gt));
                }
                if (includeAll || !query.getAnnotationIndexQuery().getPopulationFrequencyFilter().isNoOp()) {
                    scan.addColumn(family, SampleIndexSchema.toAnnotationPopFreqIndexColumn(gt));
                }
                if (includeAll || !query.getAnnotationIndexQuery().getClinicalFilter().isNoOp()) {
                    scan.addColumn(family, SampleIndexSchema.toAnnotationClinicalIndexColumn(gt));
                }
                if (includeAll || !query.emptyFileIndex()) {
                    scan.addColumn(family, SampleIndexSchema.toFileIndexColumn(gt));
                }
                if (includeAll || query.isIncludeParentColumns()
                        || query.hasFatherFilter() || query.hasMotherFilter() || query.getMendelianErrorType() != null) {
                    scan.addColumn(family, SampleIndexSchema.toParentsGTColumn(gt));
                }
            }
        }
        if (query.getMendelianError()) {
            scan.addColumn(family, SampleIndexSchema.toMendelianErrorColumn());
        }
        scan.setCaching(hBaseManager.getConf().getInt("hbase.client.scanner.caching", 100));

        logger.info("---------");
        logger.info("Sample = \"" + query.getSample() + "\" , schema version = " + query.getSchema().getVersion());
        printScan(scan);
        printQuery(locusQuery);
        printQuery(query);

//        try {
//            System.out.println("scan = " + scan.toJSON() + " " + rowKeyToString(scan.getStartRow()) + " -> + "
// + rowKeyToString(scan.getStopRow()));
//        } catch (IOException e) {
//            throw VariantQueryException.internalException(e);
//        }

        return scan;
    }

    public static void printQuery(SampleAnnotationIndexQuery annotationIndexQuery) {
        logger.info("AnnotationIndex = " + IndexUtils.maskToString(
                annotationIndexQuery.getAnnotationIndexMask(), annotationIndexQuery.getAnnotationIndex()));
        if (!annotationIndexQuery.getBiotypeFilter().isNoOp()) {
            logger.info("Biotype filter  = " + annotationIndexQuery.getBiotypeFilter().toString());
        }
        if (!annotationIndexQuery.getConsequenceTypeFilter().isNoOp()) {
            logger.info("CT filter       = " + annotationIndexQuery.getConsequenceTypeFilter().toString());
        }
        if (!annotationIndexQuery.getTranscriptFlagFilter().isNoOp()) {
            logger.info("Tf filter       = " + annotationIndexQuery.getTranscriptFlagFilter().toString());
        }
        if (!annotationIndexQuery.getCtBtTfFilter().isNoOp()) {
            logger.info("CtBtTf filter     = " + annotationIndexQuery.getCtBtTfFilter().toString());
        }
        if (!annotationIndexQuery.getClinicalFilter().isNoOp()) {
            logger.info("Clinical filter = " + annotationIndexQuery.getClinicalFilter());
        }
        if (!annotationIndexQuery.getPopulationFrequencyFilter().isNoOp()) {
            logger.info("PopFreq filter  = " + annotationIndexQuery.getPopulationFrequencyFilter());
        }
    }

    public static void printQuery(SampleIndexQuery query, Query variantsQuery) {
        printQuery(query);

        List<String> nonCoveredParams = VariantQueryUtils.validParams(variantsQuery, true)
                .stream().map(VariantQueryParam::key).collect(Collectors.toList());
        logger.info("Non covered params : " + nonCoveredParams);
    }

    public static void printQuery(SampleIndexQuery query) {
        printQuery(query.getAnnotationIndexQuery());
        logger.info("Study  : " + query.getStudy());
        if (CollectionUtils.isNotEmpty(query.getLocusQueries())) {
            List<Region> regions = query.getAllRegions();
            if (!regions.isEmpty()) {
                if (regions.size() > 10) {
                    logger.info("Regions  : #" + regions.size()
                            + " [ " + regions.get(0) + " , " + regions.get(1) + " .... , " + regions.get(regions.size() - 1) + " ] ");
                } else {
                    logger.info("Regions  : #" + regions.size() + " " + regions);
                }
            }
            List<Variant> variants = query.getAllVariants();
            if (!variants.isEmpty()) {
                if (variants.size() > 10) {
                    logger.info("Variants  : #" + variants.size()
                            + " [ " + variants.get(0) + " , " + variants.get(1) + " .... , " + variants.get(variants.size() - 1) + "] ");
                } else {
                    logger.info("Variants  : #" + variants.size() + " " + variants);
                }
            }
        }

        Iterator<String> iterator = query.getSamplesMap().keySet().iterator();
        while (iterator.hasNext()) {
            String sample = iterator.next();
            logger.info("  Sample : " + sample);
            printSingleSampleIndexQuery(query.forSample(sample), true);
            if (iterator.hasNext()) {
                logger.info("SampleIndex " + query.getQueryOperation().name());
            }
        }
    }

    public static void printScan(Scan scan) {
        logger.info("StartRow = " + Bytes.toStringBinary(scan.getStartRow()) + " == "
                + SampleIndexSchema.rowKeyToString(scan.getStartRow()));
        logger.info("StopRow = " + Bytes.toStringBinary(scan.getStopRow()) + " == "
                + SampleIndexSchema.rowKeyToString(scan.getStopRow()));
        logger.info("columns = " + scan.getFamilyMap().getOrDefault(family, Collections.emptyNavigableSet())
                .stream().map(Bytes::toString).collect(Collectors.joining(",")));
//        logger.info("MaxResultSize = " + scan.getMaxResultSize());
//        logger.info("Filters = " + scan.getFilter());
//        logger.info("Batch = " + scan.getBatch());
        logger.info("Caching = " + scan.getCaching());
    }

    public static void printQuery(LocusQuery locusQuery) {
        if (locusQuery != null) {
            logger.info("ChunkRegion: [ " + locusQuery.getChunkRegion() + " )");
            if (!locusQuery.getRegions().isEmpty()) {
                logger.info("  - Regions: " + locusQuery.getRegions());
            }
            if (!locusQuery.getVariants().isEmpty()) {
                logger.info("  - Variants: " + locusQuery.getVariants());
            }
        }
    }

    public static void printQuery(SingleSampleIndexQuery query) {
        printQuery(query.getAnnotationIndexQuery());
        printSingleSampleIndexQuery(query, false);
    }

    private static void printSingleSampleIndexQuery(SingleSampleIndexQuery query, boolean tab) {
        Iterator<SampleFileIndexQuery> iterator = query.getSampleFileIndexQuery().iterator();
        while (iterator.hasNext()) {
            SampleFileIndexQuery sampleFileIndexQuery = iterator.next();
            for (IndexFieldFilter filter : sampleFileIndexQuery.getFilters()) {
                logger.info((tab ? "      " : "") + "Filter       = " + filter);
            }
            if (iterator.hasNext()) {
                logger.info((tab ? "    " : "") + "FileIndex " + query.getSampleFileIndexQuery().getOperation());
            }
        }
        if (query.hasFatherFilter()) {
            logger.info((tab ? "    " : "") + "FatherFilter       = " + IndexUtils.parentFilterToString(query.getFatherFilter()));
        }
        if (query.hasMotherFilter()) {
            logger.info((tab ? "    " : "") + "MotherFilter       = " + IndexUtils.parentFilterToString(query.getMotherFilter()));
        }
    }

    private int toSampleId(int studyId, String sample) {
        return metadataManager.getSampleId(studyId, sample);
    }

    public boolean createTableIfNeeded(int studyId, int version, ObjectMap options) {
        String sampleIndexTable = getSampleIndexTableName(studyId, version);
        try {
            if (hBaseManager.tableExists(sampleIndexTable)) {
                return false;
            }
        } catch (IOException e) {
            throw new UncheckedIOException(e);
        }
        int files = options.getInt(
                HadoopVariantStorageOptions.EXPECTED_FILES_NUMBER.key(),
                HadoopVariantStorageOptions.EXPECTED_FILES_NUMBER.defaultValue());
        int samples = options.getInt(
                HadoopVariantStorageOptions.EXPECTED_SAMPLES_NUMBER.key(),
                files);
        // Check actual indexed samples size.
        // This value might be larger than "expected samples" if the table is being rebuilt.
        int indexedSamplesSize = metadataManager.getIndexedSamples(studyId).size();
        samples = Math.max(samples, indexedSamplesSize);

        int preSplitSize = options.getInt(
                HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_PRESPLIT_SIZE.key(),
                HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_PRESPLIT_SIZE.defaultValue());

        int splits = samples / preSplitSize;
        ArrayList<byte[]> preSplits = new ArrayList<>(splits);
        for (int i = 0; i < splits; i++) {
            preSplits.add(SampleIndexSchema.toRowKey(i * preSplitSize));
        }

        Compression.Algorithm compression = Compression.getCompressionAlgorithmByName(
                options.getString(
                        HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_COMPRESSION.key(),
                        HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_COMPRESSION.defaultValue()));

        try {
            return hBaseManager.createTableIfNeeded(sampleIndexTable, GenomeHelper.COLUMN_FAMILY_BYTES, preSplits, compression);
        } catch (IOException e) {
            throw new UncheckedIOException(e);
        }
    }

    public void updateSampleIndexSchemaStatus(int studyId, int version) throws StorageEngineException {
        StudyMetadata studyMetadata = metadataManager.getStudyMetadata(studyId);
        if (studyMetadata.getSampleIndexConfiguration(version).getStatus()
                != StudyMetadata.SampleIndexConfigurationVersioned.Status.STAGING) {
            // Update only if status is STAGING
            return;
        }
        boolean allSamplesWithThisVersion = isAllSamplesWithThisVersion(studyId, version);
        if (allSamplesWithThisVersion) {
            metadataManager.updateStudyMetadata(studyId, sm -> {
                sm.getSampleIndexConfiguration(version)
                        .setStatus(StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE);
            });
        } else {
            logger.info("Not all samples had the sample index version {} on GENOTYPES and ANNOTATION", version);
        }
    }

    public void deprecateSampleIndexSchemas(int studyId) throws StorageEngineException {
        StudyMetadata studyMetadata = metadataManager.getStudyMetadata(studyId);
        Set<Integer> deprecatables = new LinkedHashSet<>();
        for (StudyMetadata.SampleIndexConfigurationVersioned sampleIndexConfiguration : studyMetadata.getSampleIndexConfigurations()) {
            if (sampleIndexConfiguration.getStatus() == StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE
                    || sampleIndexConfiguration.getStatus() == StudyMetadata.SampleIndexConfigurationVersioned.Status.STAGING) {
                deprecatables.add(sampleIndexConfiguration.getVersion());
            }
        }
        StudyMetadata.SampleIndexConfigurationVersioned latest = studyMetadata.getSampleIndexConfigurationLatest(false);
        if (latest == null) {
            logger.info("No active sample index available for study '{}'", studyMetadata.getName());
            return;
        }
        int latestVersion = latest.getVersion();
        deprecatables.remove(latestVersion);
        if (deprecatables.isEmpty()) {
            logger.info("No sample indexes to deprecate for study '{}'", studyMetadata.getName());
            return;
        }

        int totalSamples = 0;
        int samplesNotInLatest = 0;
        for (SampleMetadata sampleMetadata : metadataManager.sampleMetadataIterable(studyId)) {
            // Only check indexed samples
            if (sampleMetadata.getIndexStatus() == TaskMetadata.Status.READY) {
                totalSamples++;
                if (!sampleWithThisVersion(sampleMetadata, latestVersion)) {
                    samplesNotInLatest++;
                }
            }
        }
        if (samplesNotInLatest > 0) {
            logger.warn("Unable to deprecate sample indexes. {} samples out of {} not in the latest sample index for study '{}'",
                    samplesNotInLatest, totalSamples, studyMetadata.getName());
            return;
        }
        logger.info("Deprecate sample indexes {} for study '{}'", deprecatables, studyMetadata.getName());
        metadataManager.updateStudyMetadata(studyId, sm -> {
            for (Integer version : deprecatables) {
                sm.getSampleIndexConfiguration(version).setStatus(StudyMetadata.SampleIndexConfigurationVersioned.Status.DEPRECATED);
            }
        });
    }

    private boolean isAllSamplesWithThisVersion(int studyId, int version) {
        boolean allSamplesWithThisVersion = true;
        for (SampleMetadata sampleMetadata : metadataManager.sampleMetadataIterable(studyId)) {
            // Only check indexed samples
            if (sampleMetadata.getIndexStatus() == TaskMetadata.Status.READY) {
                allSamplesWithThisVersion = sampleWithThisVersion(sampleMetadata, version);
                if (!allSamplesWithThisVersion) {
                    break;
                }
            }
        }
        return allSamplesWithThisVersion;
    }

    private boolean sampleWithThisVersion(SampleMetadata sampleMetadata, int version) {
        // Check SampleIndex + SampleIndexAnnotation + FamilyIndex (if needed)
        if (sampleMetadata.getSampleIndexStatus(version) != TaskMetadata.Status.READY) {
            return false;
        }
        if (sampleMetadata.getSampleIndexAnnotationStatus(version) != TaskMetadata.Status.READY) {
            return false;
        }
        if (sampleMetadata.isFamilyIndexDefined()) {
            if (sampleMetadata.getFamilyIndexStatus(version) != TaskMetadata.Status.READY) {
                return false;
            }
        }
        return true;
    }

}

redpajama_set_name: RedPajamaGithub


The graceful lines and large scale of this classic Barquentine is sure to be an eye catcher. With the eleven engraved paper sails, this traditional masterpiece is a timeless addition to any nautical collection.
The framed size is 17.5" wide by 13.5" high.


"Very few things inspire us like space. It can be easy to forget what's over us as we go about our daily lives, but then every night the curtains are lifted and we see this vast unlimited space.
Galaxies, stars, planets, moons, asteroids and space dust, unlimited opportunity to further ourselves in our understanding and our capabilities, the notion of being able to leave the surface of our planet it is a relatively new one.
The Jet Age was upon us. And the spruce and fabric of the Wright fire had given way to advanced air space alloys. And the chain and sprocket propellers had given way to turbine jet engines and suddenly global transportation on a time scale that had never been possible before became a reality.
Something else happened that year.As you all know we walked on the moon. Well. He walked on the moon. But the significance of that was clear to everyone watching at home on earth. And we had built the biggest rocket ever and it worked. Which is amazing and with this, with the success behind us what would happen next.
As the rocket continue to evolve. What could we do? We could go to Mars and build a colony there. We could put solar panels in outer space where there are never clouds and there's never night.
And we could beam sustainable energy from space to Earth. Twenty four hours a day. We could build the biggest telescope ever, bigger than any telescope ever built or even proposed to be built on earth, in outer space where there's no gravity and no atmosphere and we could we be able to not only detect the presence of exoplanets, planets of other Suns but actually see what they look like.
We could do all this and more as the rocket of evolved and indeed. Since then we have done amazing things we have put telescopes in space, we've have space stations. And all of these achievements are amazing. And we've gotten better at making rockets.
For example, space X. is now leading the way. And changing what we think about the cost and performance of rockets. But the rocket itself has not fundamentally changed. Like the rockets of the Apollo program the rockets we used today are powered by chemical combustion, on using chemical propellant and despite years of work, the cost of space launch has remained tremendously expensive, too expensive for all but a handful of the opportunities that are available in space. So why has have chemical rockets remained so expensive. Well for one reason, they push our abilities to engineer and produce materials to the limits.
As an example, the space shuttle main tank holds eight hundred tons of liquid hydrogen and liquid oxygen, a tremendously dangerous combination, and holds this of over several gravity acceleration, all the way from the surface of the earth to space. And yet has a mass fraction of the contents. The space shuttle tank weighs less than an aluminium soda can. If cars were made with this level of materials and engineering< a full size pick-up truck would weigh less than one hundred pounds. Another challenge is that chemical rockets, due to their efficiency limitations, require multiple stages and are generally non-reusable.
Imagine if, just to get to work every day you had to ditch your car two or three times to go buy another one. And that's why it's so exciting to see Space X. making progress in this area of reusable rockets that's literally as we speak. But another fundamental challenge is that throughout the history of space flight it's been risky.
Chemical propellants are incredibly energetic, very dangerous. And unfortunately over engineering isn't an option. You can't afford the weight that it takes to make a component any stronger than it absolutely needs to be. And walking this fine line sometimes mistakes are made.
Could the space shuttle Challenger disaster have been prevented if there were three O. rings instead of only two? So to understand a little bit about why chemical rockets are limited in their performance and how we can get around this will take a very brief course in rocket science.
I'm not a rocket scientist and I apologize to the actual rocket scientists for this demonstration but for the rest of us.Rockets work on a principle that we're all very familiar with. For every action there's an equal and opposite reaction. The action is a propellant being expelled by the nozzle of the rocket.
And the reaction is the acceleration of the rocket. So it's tempting to think that to improve efficiency of a rocket or to make it better we'll make it expel more propellant so that it goes faster. But it turns out that when we do this we have to build a bigger rocket because we have to hold more propellant. And the increased force that we gain by expelling more propellant is offset by the fact that we have a heavier slower vehicle and a larger rocket that we need to accelerate.
So the real way to improve the efficiency of a rocket is to increase the speed with which the propellant comes out of the rocket. This effective exhaust velocity is the key metric and rocket Scientists call this number, The specific impulse or I.S.P.. And it's like. It's the mpg of rockets, not exactly but for the purpose of our discussion.
So if a rocket had a window sticker like a car which I'm pretty sure they don't. It would look like this. And this would correspond to the space shuttle engine which is regarded as is one of the most efficient engines made, all its Pacific impulse is four hundred fifty seconds and why. Why can't it be higher? Well this engine works by combusting hydrogen and oxygen on these or chemical bonds that are have certain energies.
And if you do the math, I didn't but someone else did, the fundamental theoretical maximum energy available to these propellant would give you a theoretical I.S.P. of five hundred seconds.
So the fact of the space shuttle engine can get four hundred and fifty is actually pretty good it did. It did pretty good. But how do we get it higher? And why does it matter? The I.S.P. determines how much you can actually carry with you to space.
So let's assume that we have a rocket. No matter how much it weighs five percent of its mass will be the structure, the mechanical support that holds a rocket together. That gives us ninety five percent of the mass to be either Payload or propellant. Obviously we want to get as much payload to space as we can. What we see is if the specific impulse is below around three hundred seconds in this example.
Then we can't carry any payload to space, the rocket is not going to space at all. Now with the range of a specific impulse achievable with chemical engines around four hundred and slightly above, we see that it's possible to carry a few percent Payload to space. And that's why getting the space is so difficult today it's only barely possible.
We can afford to make the rocket any heavier. It barely works at all but if we could improve the specific impulse to a modest number let's say it can be doubled. To eight hundred seconds. We see that the payload increases dramatically in this case by ten times. And that's why it's so interesting and so exciting to work on a fundamentally new technology for space launch.
Now. As I said the best known propellant, which are generally regarded as hydrogen and oxygen, are limited in their specific impulse available. So even at thirty six hundred degrees the incredibly hot exhaust that comes out of a hydrogen oxygen engine, these mostly steam molecules are not moving fast enough. So what we can do is instead of trying to get the exhaust to be hotter, we can use a lighter weight exhaust molecule.
Hydrogen is a very light molecule and if we can heat it to only eight hundred degrees. It will be moving so fast that it will correspond to a specific impulse of eight hundred to a thousand seconds. Twice as good as steam even though it's a much cooler temperature and these are temperatures that are compatible with known materials.
So the technology that can be used to this propellant, Hydrogen, is something that we actually use every day in our kitchen to heat our food microwave energy. By beaming microwave energy from the ground to the vehicle to heat the hydrogen, we can efficiently propel a rocket to space. Now this idea was actually conceived over one hundred years ago by one of the grandfathers of rocketry.
But more recently in the two thousands, work at Cal Tech, NASA and J.P.L. it was done to show that this would become feasible using today's technologies. And most recently a set-up company was founded by students from Cal Tech. Dmitriy Tseliakhovich, Shaun McGuire and was supported by Richard Schaden, founded Escape Dynamics; which is the companies that I work at.
And we have the support of Cal Tech., Of Auto-desk and Singularity University. And we're putting together a team to bring this concept to reality. Using today's technology. So what's remarkable about this idea is that the rocket can be much simpler than chemical combustion rocket engine.
What we need is a tank to hold the hydrogen and below that is a heat exchanger, this is what absorbs the microwave energy as hydrogen flows through this and heats it up for the temperatures needed. So that when it passes through the nozzle it will be converted efficiently to thrust.
Now of course an actual vehicle will be much more complex than this. But the fundamental components ultimately are much simpler. So to make this a reality. We're working to utilize existing technologies and demonstrate them and integrate them at a scale that hasn't been achieved yet.
So we're working on making high power hundred kilowatt microwave sources called gyrotrons and will be scaling these up to megawatt output capacity to provide the ground based power needed for this propulsion system.
We're developing advanced high temperature materials for the heat exchanger. That will be required both to withstand the great stresses of high gas pressures on an aerodynamic loading, while still also tolerating the high temperatures that will be involved with this. And we're working on advanced beaming and tracking systems on the ground to enable safe reliable delivery of Microwave energy to a flying vehicle.
We've been working for over two years now on the safety systems for these to make sure that we can make it safe for people. And yes even for birds to be prevented from exposure to this through safety interlocks. Now there's another very exciting application of this technology. And it's something that will be able to demonstrate much sooner than will be able to demonstrate space launch. And that is.
We will be able to use this ground based infrastructure to deliver wireless electrical power to AVI's, to aircraft's flying through earth's atmosphere and enabling technology for this is the RecTena which is a combination for rectifying antenna. Something that was invented over fifty years ago by William Brown at Raytheon and working with the team at NASA and J.P.L..
He was able to demonstrate that microwave power can be beamed very efficiently over great distances over miles. And now we're ready to put this technology to work. So by a coupling this technology with modern UABs which are evolving every day for an interesting number of applications. We're going to be able to put together U.A.B. systems that can fly and recharge in the air without ever needing to land. So it's not going to be easy to bring this vision to reality.
But we think it's going to be fun and it's going to be worth it. So I want within my lifetime to see us go farther than the moon. I would want to see us go to Mars and build the biggest telescope yet.
Let's go mine asteroids. Or leave our solar system. It's going to be exciting. And it's going to start with finding a better way to go to space.


When my then-good friend (later boyfriend) showed me how to play HeroClix in September of 2007, he helped me build a team out of his collection to play my first game against him. I picked out the Limited Edition Sue Storm–I didn't know much about her, but I already liked her 19 defense, which I could already see was rare among Clix figures of her point cost (49).
I already liked high defenses in the other games I played, so it didn't take me long to gather some high-defense figures in Clix, and Sue was one of the first I got when I could. She was one of my first favorite Clix pieces because of that fantastic 19 defense, and she's still quite playable today.
LE Sue Storm, #205 out of Fantastic Forces. Image Credit: HCRealms.com.
With Stealth, Barrier, and 19 defense first click, it's a good bet Sue won't be targeted or damaged easily, even with today's higher attacks. And even if she does get hit, her defense drops slowly, point by point (until her last click when it drops two points). Pair this with her Flight ability, and she's a solid taxi for grounded friendly characters, as well as a darn near immovable block if you need to keep enemy characters away from wounded friendlies.
Now, most people pooh-pooh the back half of her dial, and some don't think much of her offensive abilities in general. But I wouldn't be so quick to judge. Her 9 attack and 2 damage up-front can be good in a pinch if you need her as a second- or third-string attacker, and Outwit or Perplex can always make her shot easier. Don't underestimate that Incapacitate and Energy Shield/Deflection on her back-dial, either–I've had LE Sue Storm be the last to survive of my team on next-to-last click, and still be making fine shots at my opponent's banged-up team.
These days, that kind of team is no longer valid; you now generally play a Wildcard Fan Four ATA team like a Defend team, surrounding Sue with your wildcards so they can borrow her defense while they attack. You also have to pay 5 points from your team build cost for using the ATA on a "Wildcards & Sue" team. Still useful and successful, though not as mobile as the old one. (The old one was rather broken, I'll admit. This one's more fun for player and opponent both.) I actually like this new build pretty well, because surrounding Sue with little Wildcards who are both protected by Sue and guarding her is FUNNY.
If you love Stealth pieces, and love pieces with high defenses, then this Sue should be in your collection. For 49 points, you won't find another 19-defense Flight piece that has this kind of team support (like Barrier and Incapacitate) built in. She's a great addition to any long-game Clix player's arsenal.


This workshop is a free online webinar for main street small businesses in Ontario. Participate online for free, using your preferred device (laptop, smartphone or tablet). All you need is a fast internet connection and free webinar software. Download details will be provided after registration.