In prepartion before I replace my trusty sb3 with a Touch.
The touch will be primarily used with a usb hd directly.
This will mean I can sell my desktop,monitor etc.
Leaving me and the Mrs with a 17" latop and web book to share.
Currently I control the SB3 with my Logitech Harmony 525 remote.
I've recently installed a modded version of Ipeng on my Nokia 5800.
Which is quite nice but not perfect.
Is Ipeng on the Iphone and Ipod touch identical and nice and smooth, on the Nokia not all the icons etc are correct.
As my mobile contract is up soon is the Iphone a serious contender for a new phone ? Or will an android version of Ipeng be developed ?
The things that put me off the Iphone are the rip off tariffs here in the UK.
Is the build quality better on the Ipod Touch.
As both Apple devices have integrated batteries is it a return to Apple to have it replaced ?
Finally would the Ipeng app been installed directly on the Touch in my scenario ?
There are two versions of iPeng available.
The free iPeng skin that you've used on the Nokia and the iPeng App.
The App will only run on an iPhone/iPod but is much better, because it is a real native app instead of "just" a web page.
The App has more features and is much smoother than the skin.
There are several SqueezeBox Apps for Android available.
No, you do not need to install anything on the Touch.
AFAIK, the SB Touch will not include a WebUI. That means you will not be able to install the iPeng skin.
But it does not affect the iPeng App (or other native SB Apps) as it communicates directly with your server/Touch, not through the WebUI.
As my mobile contract is up soon is the Iphone a serious contender for a new phone ?
iPhone is always a serious contender for a new Phone.
Buy an import from Italy or France. Factory unlocked without limitations. And then get a good tariff.
Can't agree. Build quality is not too good but still better than on the various htc phones I have (hero, touch pro, MDA III). Will work for two years (my hero recently started to fall apart after 9 months but then that phone, 9 months after showing up on the market is already 4 software generation behind without an update so needs to be replaced anyway - this is a STRONG point for iPhone: the software lasts). No longer but then since my venerable 6310i no phone has worked longer for me.
Reception isn't bad for me (iPhone 3G), comparable to other phones.
That said, there are better PHONES out there than iPhone, yet no better smartphones, IMHO. Android's catching up, though.
Battery lasts longer than for all the competitors I've tested so far (an old SonyEricsson M600i, htc hero, htc touch pro).
Feels good, latest one has some issues with me (I've got three, the older ones are fine).
well....there are stores that can replace them, at least here in Berlin.
The iPeng APP goes onto the iPhone or iTouch. The iPeng PLUGIN won't work on the SB Touch.


The all new TexSpray HTX 2030 is Gracos most powerful and versatile texture sprayer to date. Designed for todays demanding exterior aggregate and smooth coatings, the HTX 2030 offers extreme durability and simple operation. Capable of applying exterior smooth and aggregate textures, elastomerics, and acrylic materials, it can also spray drywall mud in an orange peel, splatter or knockdown finish.


text:
Get access to Strub family records.Start a free trial
Strub Family History
Strub Name Meaning
German: nickname for someone with stiff, bristly hair, from Middle Low German struf, Middle High German strup 'unkempt', 'rough'. Compare Struve.
Similar surnames: Straub, Staub, Strupp, Strutz, Stroh, Taub, Strunk, Struve, Traub
45 thousand record(s) for Strub
Where is the Strub family from?
You can see how Strub families moved over time by selecting different census years. The Strub family name was found in the USA, the UK, Canada, and Scotland between 1861 and 1920. The most Strub families were found in the USA in 1880. In 1880 there were 31 Strub families living in Illinois. This was about 15% of all the recorded Strub's in the USA. Illinois had the highest population of Strub families in 1880.
Use census records and voter lists to see where families with the Strub surname lived. Within census records, you can often find information like name of household members, ages, birthplaces, residences, and occupations.
View Census Data for Strub
What did your Strub ancestors do for a living?
In 1940, Farmer and Bookkeeper were the top reported jobs for men and women in the US named Strub. 17% of Strub men worked as a Farmer and 13% of Strub women worked as a Bookkeeper. Some less common occupations for Americans named Strub were Truck Driver and Housewife.
View Census data for Strub | Data not to scale
What Strub family records will you find?
There are 6,000 census records available for the last name Strub. Like a window into their day-to-day life, Strub census records can tell you where and how your ancestors worked, their level of education, veteran status, and more.
Search US census records for Strub
There are 1,000 immigration records available for the last name Strub. Passenger lists are your ticket to knowing when your ancestors arrived in the USA, and how they made the journey - from the ship name to ports of arrival and departure.
View all Strub immigration records
There are 1,000 military records available for the last name Strub. For the veterans among your Strub ancestors, military collections provide insights into where and when they served, and even physical descriptions.
View all Strub military records
You've only scratched the surface of Strub family history.
What is the average Strub lifespan?
Between 1959 and 2004, in the United States, Strub life expectancy was at its lowest point in 1959, and highest in 2004. The average life expectancy for Strub in 1959 was 50, and 86 in 2004.
View Social Security Death Index (SSDI) for Strub
An unusually short lifespan might indicate that your Strub ancestors lived in harsh conditions. A short lifespan might also indicate health problems that were once prevalent in your family. The SSDI is a searchable database of more than 70 million names. You can find birthdates, death dates, addresses and more.
Discover the unique achievements of ancestors in the Strub family tree

redpajama_set_name: RedPajamaCommonCrawl


Atlantic Pacific provides complete end-to-end supply chain management solutions to the defence industry. Our employees within our 'defence' division have experience and knowledge of the complex regulations and strict compliance regimes within this industry sector.
We provide scalable logistics solutions to companies in this sector incorporating freight forwarding, project management, warehousing, packaging design and reverse logistics.
Atlantic Pacific provides 24/7 customer service and operational support for our customers within the Defence sector.
We are also members of ADS (Aerospace, Defence and Security) organisation and signatories to their Supply Chain 21st Century (SC21) programme.


free tutorial for tiny tree ornaments!
As a little early holiday gift, here's a tutorial for how to make your own tiny tree ornaments. These little guys are a great way to use up yarn scraps, or feature little bits of hand-spun yarn. They knit up quickly, and make great gift toppers, too!
1. Loosely cast on 3 stitches in trunk color.
2. Knit 5 rows in trunk color.
3. Leaving a 12" tail, cast on 4 stitches in main color.
4. Knit the 3 trunk stitches as follows: knit first stitch using tail, knit second stitch using yarn from the ball, and the third stitch using the tail yarn.
5. Cast on 4 stitches in main color. You should have 11 stitches.
Row 6: k2tog, k7, k2tog. 9 stitches remain.
Row 12: k2tog, k5, k2tog. 7 stitches remain.
Row 18: k2tog, k3, k2tog. 5 stitches remain.
Rows: 19 – 23: knit.
Row 24: k2tog, k1, k2tog. 3 stitches remain.
Row 25 - 27: knit.
Row 28: k3tog. Cut main color, pass tail through last remaining stitch.
7. Weave in ends. Gently steam your finished tree using an iron on the wool setting. Decorate as desired!


The InputType Enum controls the SurfaceInputs of Part. Several parameters here are left-overs from 2005, before Roblox was a multiplayer game, and have no known functionality.
Behaves like a weld, and does absolutely nothing.
Rotate at a constant velocity of /BasePart ParamB.
ParamA * math.sin(game.Workspace.DistributedGameTime * ParamB).
/BasePart ParamA will determine the maximum speed at which the part will spin, and BasePart ParamB will determine how frequently it changes direction.


text:
Springsteen's Long, Long Overdue Guitar Album
High Hopes, featuring Tom Morello from Rage Against the Machine.
By Ryan D'Agostino
When Bruce Springsteen appeared on the covers of Time and Newsweek in the same week in 1975, he told the Newsweek reporter, "The first day I can remember lookin' in the mirror and standin' what I was seeing was the day I had a guitar in my hand."
Springsteen loves the guitar. In concert, he is a prolific and operatic lead man. (He is number 96 on Rolling Stone's list of the top 100 guitarists of all time.) The E Street Band includes three guitarists: himself, Steven Van Zandt, and Nils Lofgren. And yet Springsteen's seventeen studio albums have never been guitar showcases. He wails on "Jungleland," and "Candy's Room," sure, but all in all there are few big solos, and nothing approaching what he does live.
Until now, the release of his eighteenth record, High Hopes. The album's muse (Springsteen's word) is fierce guitar player Tom Morello (40 on the RS list), whose former band Rage Against the Machine turned Springsteen's 1996 acoustic ballad "The Ghost of Tom Joad" into a thunderous anthem. On High Hopes Morello joins the band (guitarist number four) for eight songs—including "Tom Joad," on which his solo is historic. His guitar propels the title track into outer space. Another, "41 Shots," was written after the 1999 shooting of Amidou Diallo by New York City police—last year in concert Springsteen dedicated it to Trayvon Martin. It gets a proper recording now, with Morello driving.
A few songs you might listen to politely then set aside—"Harry's Place" sounds like what it feels like to watch an episode of Miami Vice, and "Just Like Fire Would" is kind of weird. But for the most part, Springsteen is once more a man standing with a guitar in his hand, triumphant.
Follow The Culture Blog on RSS and on Twitter at @ESQCulture.
Ryan D'Agostino Editor-in-Chief Ryan D'Agostino is the Editor-in-Chief of Popular Mechanics.
The Abortion Ministry of Dr. Willie Parker
The Drugging of the American Boy
Bitcoin Wound Up Being Just As Broken as the Corrupt Banking System It Was Trying to Kill
Penélope Cruz Is the Sexiest Woman Alive


text:
Taglit-Birthright Israel Launches 'Innovation Center' in Tel Aviv
by JNS.org
Visitors to the Birthright Israel State of Mind Innovation Center. Photo: Facebook.
JNS.org — An educational tourism organization that offers young adults free trips to Israel has opened an "innovation center" in Tel Aviv, in collaboration with the city's stock exchange, to showcase the Jewish state's contributions to entrepreneurship, innovation and technology worldwide.
Taglit-Birthright, which has to date brought more than half a million Jews aged 18-26 on free first-time, 10-day trips to Israel, opened the Birthright Israel State of Mind Innovation Center in the presence of some 200 business leaders, entrepreneurs and philanthropists Wednesday night.
"Birthright is the greatest start-up that was established by the state of Israel and the Jewish people, both in terms of quality and quantity," Birthright Israel CEO Gidi Mark said at the event. "Before Birthright Israel was founded, 17 years ago, 2,000 students arrived to Israel each year; and through Birthright Israel, nowadays, tens of thousands of young Jews from around the world visit Israel annually."
The innovation center features an exhibition highlighting Israel's contributions to agriculture, medicine, cybersecurity, science, transportation and aerospace, along with a series of lectures from successful Israeli entrepreneurs.
This summer, the center ran a pilot season that saw visits from more than 13,000 Birthright Israel participants and dozens of delegations from Israel and around the world. The center expects to welcome more than 45,000 visitors in 2017.


text:
Chicago: Frank Lloyd Wright 3
We now turn to the studio that Wright constructed in front of his house in Oak Park. The original house dates to 1889: the studio and connecting corridor were built in 1898.
"Most of the sculptures on the exterior of the Home and studio were designed by Wright's friend and collaborator, Richard Bock. These include the two boulder figures flanking the entrance of the studio, which features a man crouching and breaking free from the ground beneath him. Bock also designed the stork capitals on the exterior loggia of the studio. The capitals signifies the tree of life, the book of knowledge, an architectural scroll, and two storks full of wisdom and fertility." (wikipedia)
The studio has lots of natural light thanks to the skylight with its characteristic pattern of coloured glass. The gift shop has, of course, lots of things with patterns like this on them.
I was a bit surprised that the drawings were all laid out on flat tables – which could be adjusted for height. Most of the popular images for architects drawing boards show them tilted at an angle.
The guide did a bit of audience involvement here. He got four people to hold hands as they stood around him, and then move back until their arms were extended: then he told them to lean out further. The roof of the octagonal studio is supported by the metal frame and chains, and the four people could all testify to how that works.
Model of the Robie House
I imagine that the secretary who had to sit in this chair as she did Wright's typing has some thoughts about his lack of understanding of human anatomy and ergonomics.
The next post in this series will deal with Wright's first commission for a public building.
951 Chicago Ave, Oak Park, IL 60302, USA
Posted in architecture
« Chicago: Frank Lloyd Wright 2
Chicago: Frank Lloyd Wright 4 »


Up for some Metal Gear goodies? Well, here's a round up of all MGS merchandise related pictures shared by Kojima (among others) in the past days. First, the Metal Gear Solid Deluxe Edition, a re-release of the comics by Ashley Wood.
Secondly, some detailed shots of the head belonging to the Play Arts Ground Zeroes Snake that was on display at TGS.
A MGSV: Ground Zeroes case for iPhone 6.
A Diamond Dogs branded cushion, it's not clear whether this will be sold in stores.
Some new pictures of the Solid Snake and Gray Fox RevolMini figures, including the packaging.
Prototype of the sneaking boots by Puma, unveiled at TGS.
Lastly, two pictures that were posted to RevolMini's official Twitter account showing Venom Snake.


Save more save with 123CopyDVD promo code. By using this code, you will take 50% Off on All Orders. Hurry up, because the time is limited!
123CopyDVD Coupon Codes: Try Free Trial of 123 Media Vault. Don't miss this great chance!
With 123CopyDVD Coupon Code at your checkout, you can save 20% off!! Nothing is easier! Hurry up! Don't miss the chance!
Earn 35% Cash Back123 At Copy DVD!!
Nothing is easier! Just sign up at your checkout, you have a chance to save 35% off! Remember that the good chance never comes twice!
Apply this 123CopyDVD discount code now and get 40% Off on All Orders. Don't hesitate until too late!
Benefit 30% Off on All Orders from 123CopyDVD with this discount code. Is it great?? Get code now!
If you use this 123CopyDVD Coupon Code, you will save 20% Off on All Orders. Don't miss this great chance!
Use this coupon code now to save 10% Off Any Order from 123CopyDVD. Hurry up, because the time is limited!
Get 20% Off Any Purchase Sitewide using this 123CopyDVD Coupon Code at checkout process. Hurry up! Don't miss!
Are you looking for a chance to save your money. This is here. Using 123CopyDVD Coupon Code now to save 35% discount from 123CopyDVD.


Posted in SOC327 - Emotions Bodies and Society, Tutorial 1, Uncategorized, UOW.
← Can you imagine society?
Its an interesting question to explore, whether a society really does influence your experience of your body and feelings. I think that when i first looked at this question, i thought to myself that i was pretty in control of the way i present and view myself. But after looking at it in reference to sociology i thought of the idea that "the first wisdom of sociology is this: things are not what they seem" (Peter Berger, invitation into sociology 1963:34). This forced me to think more sociologically about this question. So yes i do agree that society influences our experience of our bodies and our feelings. After all, yes i do have a tattoo and no i can't throw a ball very well, and if i use my sociological imagination i can see that maybe i have a tattoo because they have evolved to be a social norm of todays society? Maybe i can't throw a ball well because i was born into the cyber revolution, causing me to be more exposed to technology based games than physical based games like handball?


text:
Q: WooCommerce filter search by category and custom field I want to ask how to create a woocommerce custom search form.
My search and filter form has 3 fields:
Category: I managed to pull woocommerce product category into my custom search html form with <select> tag:
<?php
$catTerms = get_terms('product_cat', array('hide_empty' => 0, 'orderby' => 'ASC', 'exclude' => '17,77'));
foreach($catTerms as $catTerm) : ?>
    <option value="<?php echo $catTerm->slug; ?>"><?php echo $catTerm->name; ?></option>
<?php endforeach; ?>

Filter By: (dropdown menu) by author:
I added extra field to woocommerce using woo_add_custom_general_fields_save() function on theme's functions.php, works properly.
Note: this is not 'custom field' we usually use on wordpress to add some more metadata, but this code below is to add more field on Product Data > General (on woocommerce).
function woo_add_custom_general_fields_save($post_id)
{
    $woocommerce_textinput = $_POST['_book_author'];
    if( !empty( $woocommerce_textinput ) )
    update_post_meta( $post_id, '_book_author', esc_html( $woocommerce_textinput ) );
}

By Title:
I managed to use this filter using http://www.example.com/wp-root/woocommerce-product-page/?s=searchproducttitle
Input text field:
This is for user to search by keyword.
So this is my complete custom html search form:
<form action="<?php echo site_url(); ?>/pm-bookstore/" method="GET">

<select name="">
    <?php $catTerms = get_terms('product_cat', array('hide_empty' => 0, 'orderby' => 'ASC', 'exclude' => '17,77')); ?>
        <?php foreach($catTerms as $catTerm) : ?>
        <option value="<?php echo $catTerm->slug; ?>"><?php echo $catTerm->name; ?></option>
    <?php endforeach; ?>
</select>

<select name="">
    <option value="">By author</option>
    <option value="">By title</option>
</select>

<input type="text" placeholder="Search Book by Title, Author, ISBN..." name="s">
<button class="fa fa-search" type="submit"></button>

</form>

For search parameter I expect to able to pull them all. But I only able to use the ?s parameter (which is only product title).
I tried using another parameter, like, product_cat, tag_ID, but no success.
At the moment I only able to use

http://www.example.com/wp-root/woocommerce-product-page/?s=searchproducttitle

My expected result is:

http://www.example.com/wp-root/woocommerce-product-page/?s=searchproducttitle&category=categoryslug&author=authorname

or

http://www.example.com/wp-root/woocommerce-product-page/?s=searchproducttitle&category=categoryslug

How to make this search parameter working on woocommerce search?
Thank you.

A: In your 'pm-bookstore' page use WP_Query to get the result.
// WP_Query arguments
$args = array (
    'name'                   => 'your title',
    'post_type'              => array( 'product' ),
    'post_status'            => array( 'publish' ),
    'tax_query'              => array(
             'taxonomy' => 'categories',
             'field'    => 'slug',
             'term'     =>  'your category slug'
     ),
    'meta_query'             => array(
        array(
            'key'       => '_book_author',
            'value'     => 'your author name',
            'compare'   => '=',
            'type'      => 'CHAR',
        ),
    ),
);

// The Query
$query = new WP_Query( $args );

I haven't tested it but it should work.

redpajama_set_name: RedPajamaStackExchange


text:
You are currently using the site but have requested a page in the site. Would you like to change to the site?
Psychological Methods" Research & Statistics
A Guide to Teaching Research Methods in Psychology
Selected type: Paperback
Bryan Saville
ISBN: 978-1-405-15481-9 January 2008 Wiley-Blackwell 238 Pages
Starting at just $55.00
A Guide to Teaching Research Methods in Psychology provides instructors with a practical handbook for teaching psychological research methods. The book
Discusses in detail many of the challenges and dilemmas that instructors often face when teaching this demanding course
Covers a wide range of topics - from pedagogical considerations and course preparation to reflection and course revision
Offers alternative strategies for teaching research methods with practical suggestions for their implementation
Accessible and informative to all current and future research methods instructors, from graduate assistants to seasoned classroom veterans
Bryan K. Saville is Assistant Professor in the Department of Psychology at James Madison University. In 2002, Saville received the McKeachie Early Career Award for excellence in teaching from the Society for the Teaching of Psychology (Division 2 of the American Psychological Association). From 2003 to 2006, he coedited E-xcellence in Teaching, a monthly e-column devoted to the teaching of psychology.
Request permission to reuse content from this site
Series Editor's Preface.
Acknowledgments.
1. An Introduction to Psychological Research Methods: Course Content and Pedagogical Considerations.
2. Getting Ready to Teach.
3. Teaching Psychology as a Science.
4. Ethical Considerations in Psychological Research: Teaching Issues.
5. Teaching Reliability and Validity.
6. Teaching the Distinction between Experimental and Nonexperimental Research Methods.
7. Teaching the Relation between Statistics and Research Methods.
8. Teaching the Distinction between Large-N and Small-N Research Designs.
9. The Laboratory as Teaching Tool.
10. Innovative Approaches to Teaching Research Methods.
11. Reflection and Revision.
Appendix: Professional Development Resources in the Teaching of Psychology.
"Saville's A Guide to Teaching Research Methods in Psychology is an invaluable resource for both younger instructors who are just starting to teach this challenging course and for more experienced instructors who are interested in fine tuning their approach to research methods. The handbook is well-written and is full of valuable ideas that are easy to incorporate into your course. I've been teaching research methods to psychology students for the past 13 years and Saville's book has truly inspired me to incorporate many of his ideas into my own course."
Jeffrey S. Anastasi, Sam Houston State University<!--end-->
"Bryan Saville has provided an immense service to those of us who teach research methods courses by offering a well-spring of advice for teaching challenging methodological topics and managing this complex course. Even for those of us who have taught methods for many years, A Guide to Teaching Research Methods in Psychology is an invaluable resource."
Mark R. Leary, Duke University
Teaching Psychological Science


There is a fifth dimension beyond that which is known to native Walker Countians. It is a dimension as vast as the space beneath a car seat, and as timeless as a mullet haircut. It is the middle ground between a fiddle and a violin, between book learning and street smarts, and it lies between the bottom of Smith Lake and the roof of the Walmart in Jasper. It is an area which we call The Bugtussle Triangle.
Earlier this year, I went on a bus trip to Montgomery with the Leadership Walker County group. In a previous blog, Lost in Montgomery, I recounted the true story of how I had to leave my red Swiss Army knife, a gift from my brother-in-law, with the security team at one of the buildings we visited and how my attempts to retrieve it that afternoon almost left me stranded in Montgomery. Thankfully, I was able to recover it and catch the bus back home.
Two weeks later, my red Swiss Army knife mysteriously vanished without a trace. One day it was on the dresser with my wallet, keys, 42 cents in loose change, two ink pens, a coupon for half off my next jug of Rafe Hollister's Mountain-Grown Moonshine, a tire gauge I found in some guy's pocket, a partially devoured rubber chicken, a pair of rusty handcuffs that allegedly belonged to Harry Houdini's third cousin's wife's great-grandmother, a manila envelope containing aerial photographs of Orson Welles, two faded ticket stubs from a 1968 midnight showing of Little Shop of Rocky Horror's Last Picture Show, a half-gallon jug of Esso gasoline, and a 50-pound Acme anvil paperweight. The next day it was gone.
I searched high and low. I formed a posse to find it. Search choppers with spotlights combed the woods near my house. My neighbor's bloodhounds came back within 4 hours, shrugging their shoulders. Oh, I've simply misplaced it, you say? Put it down somewhere absentmindedly and forgot where it is? I don't think so. The knife is merely the latest in a string of mysterious disappearances in my household which date back several years.
Socks. Shirt buttons. Coffee mugs. Reading glasses. A replica of the beautiful, elegant leg lamp in the 1983 film A Christmas Story. Assorted shirts and blue jeans. Dozens of Oreo cookies vanishing overnight from packages hidden in the cupboard. Locks of hair from the crown of my head. All missing without a trace. And these are only the documented cases compiled by our team of experts, based on some eyewitness accounts, unconfirmed hearsay, and highly circumstantial evidence. Still think these are all simply cases of absent mindedness?
In 1967, Dr. Emil Schaufhaussen, an amateur quantum physicist, unlicensed hair stylist, and small-engine-repair mechanic from Bugtussle, Alabama, conducted studies on numerous unexplained disappearances of cattle, raccoons, sheep, and farm implements in this area. He hypothesized that these disappearances were caused by a mysterious interdimensional portal located somewhere in the north central portion of this state. His theory was that the portal was somehow caused by chemicals found in Red Man Chewing Tobacco and Krispy Kreme donuts. While the existence of The Bugtussle Triangle, as it was dubbed by both of Schaufhaussen's followers, was never actually proven, it did give rise to rampant and unsubstantiated rumors which supported his hypothesis.
Perhaps one day, when someone finally proves the existence of The Bugtussle Triangle, they will send in a team to inventory the lost items. Among the missing cows, farming tools, raccoons, and other forgotten relics, maybe they will discover my red Swiss Army knife, still as dull as ever, lying next to a few dozen socks, a dozen Oreos, several pairs of reading glasses, a small lamp shaped like a leg and enough brown hair to cover a baseball.


text:
Best Books, Best US President Books, Biography & Memoir, History, Nonfiction
The Best Books To Learn About President George Washington
(You can view the rest of our presidential Best Book lists by going to our Best US President Books page, or for a more in-depth look at how we found and ranked the books you can visit our Best Book About Every United States President article.)
George Washington Quick Facts
President Number 1
Terms In Office 2
Political Party Nonpartisan
Vice President John Adams
Home State Virginia
Slaves Owned 318
Presidential Pet Donkey
Articles Used in Ranking 16
Number of Unique Books 40
The Best Book About George Washington
(Appears on 9 Lists)
Washington: A Life by Ron Chernow
Lists It Appears On:
Best Presidential Bios
Mandi Lindner
Presidents USA
Celebrated biographer Ron Chernow provides a richly nuanced portrait of the father of our nation and the first president of the United States. With a breadth and depth matched by no other one volume biography of George Washington, this crisply paced narrative carries the reader through his adventurous early years, his heroic exploits with the Continental Army during the Revolutionary War, his presiding over the Constitutional Convention, and his magnificent performance as America's first president. In this groundbreaking work, based on massive research, Chernow shatters forever the stereotype of George Washington as a stolid, unemotional figure and brings to vivid life a dashing, passionate man of fiery opinions and many moods.
Learn More About Book
#2 Book
His Excellency: George Washington by Joseph J. Ellis
At Times Dull
To this landmark biography of our first president, Joseph J. Ellis brings the exacting scholarship, shrewd analysis, and lyric prose that have made him one of the premier historians of the Revolutionary era. Training his lens on a figure who sometimes seems as remote as his effigy on Mount Rushmore, Ellis assesses George Washington as a military and political leader and a man whose "statue-like solidity" concealed volcanic energies and emotions.
Washington: The Indispensable Man by James Flexner
After more than two decades, this dramatic and concise single-volume distillation of James Thomas Flexner's definitive four-volume biography "George Washington," which received a Pulitzer Prize citation and a National Book Award for the fourth volume, has itself become an American classic. Now in a new trade paperback edition, this masterful work explores the Father of Our Country – sometimes an unpopular hero, a man of great contradictions, but always a towering historical figure, who remains, as Flexner writes in these pages, "a fallible human being made of flesh and blood and spirit – not a statue of marble and wood… a great and good man." The author unflinchingly paints a portrait of Washington: slave owner, brave leader, man of passion, reluctant politician, and fierce general. His complex character and career are neither glorified nor vilified here; rather, Flexner sets up a brilliant counterpoint between Washington's public and private lives and gives us a challenging look at the man who has become as much a national symbol as the American flag.
#4-9 Books
(Appear on 3 Lists Each)
Washington by Douglas Southall Freeman
Five Books
Washington is the most complete, definitive one-volume biography of George Washington ever written. In 1948 renowned biographer and military historian Douglas Southall Freeman won his second Pulitzer Prize for his new and dramatic reexamination of George Washington. For years biographies had gone from idolatry to muckraking in their depictions of this somewhat marbleized Founding Father. Freeman's new interpretation was a fresh step, making Washington a living, breathing individual, flawed but heroic. An able commander who defeated the British Empire against incredible odds, Washington proved to be just as adept at wielding political power, and adroitly steered our new loosely called nation through the first stormy years of our unproven federal stewardship and the first two presidential administrations.
George Washington and the New Nation 1783-1793 by James Flexner
The Tailored Man
This book begins with Washington's return to Mount Vernon, a victorious, but exhausted soldier eagerly seeking the pleasures of a quiet country life. Free of heavy responsibilities, his character expands in genial, often unexpected ways. All too soon, however, the idyll is broken. This promises to be the biography of Washington that will best serve our generation.
George Washington in the American Revolution 1775-1783 by James Flexner
History has blinded us to the all-too-human character of George Washington; in doing so, it has blinded us to the true nature of his greatness. We have urgent need to know this man we call the Father of Our Country. And now, at last, James Thomas Flexner has given us the biography that fully meets our need.
George Washington: Anguish and Farewell 1793-1799 by James Flexner
George Washington: The Forge of Experience 1732-1775 by James Flexner
This volume tells about considerably more than half George Washington's life, the forty-three years that elapsed from his birth to his acceptance, at the outbreak of the American Revolution, of the command of the Continental Army.
George Washington: A Biography by Washington Irving
Washington Irving's Life of George Washington (published in five volumes in 1856–59) was the product of his last years and remains his most personal work. Christened with the name of the great general, Irving was blessed by Washington while still a boy of seven, and later came to know many of the prominent figures of the Revolution. In these pages he describes them using firsthand source material and observation. The result is a book which is fascinating not only for its subject (the American Revolution), but also for how it reveals in illuminating detail the personality and humanity of a now remote, towering icon. Here is an intimate portrait of Washington the man, from Virginia youth to colonial commander to commander-in-chief of the patriot army to first president and great guiding force of the American federation. But one cannot read Irving's Life without marveling at the supreme art behind it, for his biography is foremost a work of literature. Charles Neider's abridgment and editing of Irving's long out-of-print classic has created a literary work comparable in importance and elegance to the original. George Washington, A Biography, Neider's title for his edition of Irving's Life, makes the work accessible to modern audiences. The extensive introduction provides a detailed analysis of Irving's life and times, and the difficulties he faced as he worked against his own failing health to finish what he felt was his masterpiece. This new edition of the superb biography of America's first citizen by America's first literary artist remains as fresh and unique today as when it was penned.
1776 by David McCullough
America's beloved and distinguished historian presents, in a book of breathtaking excitement, drama, and narrative force, the stirring story of the year of our nation's birth, 1776, interweaving, on both sides of the Atlantic, the actions and decisions that led Great Britain to undertake a war against her rebellious colonial subjects and that placed America's survival in the hands of George Washington.
Patriarch: George Washington and the New American Nation by Richard Norton Smith
A dramatic portrait of George Washington's presidential years, Patriarch is a gripping story of politics and statecraft. Smith describes Washington's struggle to preside over the bitter feud between Jefferson and Hamilton–two brilliant members of his cabinet–while attempting to distinguish the first presidency.
The Return of George Washington: 1783-1789 by Edward Larson
Pulitzer Prize-winning historian Edward J. Larson recovers a crucially important—yet almost always overlooked—chapter of George Washington's life, revealing how Washington saved the United States by coming out of retirement to lead the Constitutional Convention and serve as our first president.
Washington's Crossing by David Hackett Fischer
Six months after the Declaration of Independence, the American Revolution was all but lost. A powerful British force had routed the Americans at New York, occupied three colonies, and advanced within sight of Philadelphia.
Washington's Revolution: The Making of America's First Leader by Robert Middlekauff
Focusing on Washington's early years, Robert Middlekauff penetrates his mystique, revealing his all-too-human fears, values, and passions. Rich in psychological detail regarding Washington's temperament, idiosyncrasies, and experiences, this book shows a self-conscious Washington who grew in confidence and experience as a young soldier, businessman, and Virginia gentleman, and who was transformed into a patriot by the revolutionary ferment of the 1760s and '70s. Taking command of an army in constant dire need—of adequate food, weapons, and, at times, even clothing and shoes—Washington displayed incredible persistence and resourcefulness, growing into a leader who both understood and defined the crucial role of the army in the formation of a new American society.
#15-40 Book
(Appear on a Single List)
A Picture Book of George Washington by David A. Adler
A simple presentation of his character and the major events of his life.
An Imperfect God: George Washington, His Slaves, and the Creation of America by Henry Wiencek
When George Washington wrote his will, he made the startling decision to set his slaves free; earlier he had said that holding slaves was his "only unavoidable subject of regret." In this groundbreaking work, Henry Wiencek explores the founding father's engagement with slavery at every stage of his life–as a Virginia planter, soldier, politician, president and statesman.
Blood of Tyrants: George Washington & the Forging of the Presidency by Logan Beirne
Blood of Tyrants reveals the surprising details of our Founding Fathers' approach to government and this history's impact on today. Delving into the forgotten—and often lurid—facts of the Revolutionary War, Logan Beirne focuses on the nation's first commander in chief, George Washington, as he shaped the very meaning of the United States Constitution in the heat of battle.
First In Peace: How George Washington Set the Course for America by Conor Cruise O'Brien
First in Peace considers the dissension between Washington and Jefferson during the first U.S. presidency, and reveals Washington's clear-sighted political wisdom while exposing Jefferson's dangerous ideology. Cruise O'Brien makes the case that Washington, not Jefferson, was the true democrat, and commends his clarity of vision in restoring good relations with Britain, his preference for order and pragmatism, and his aversion to French political extremism.
Founding Father: Rediscovering George Washington by Richard Brookhiser
In this thought-provoking look at George Washington as soldier and statesman, Richard Brookhiser traces the astonishing achievements of Washington's career and illuminates how his character and his values shaped the beginnings of American politics.
Founding Friendship: George Washington, James Madison, and the Creation of the American Republic by Stuart Eric Leibiger
Although the friendship between George Washington and James Madison was eclipsed in the early 1790s by the alliances of Madison with Jefferson and Washington with Hamilton, their collaboration remains central to the constitutional revolution that launched the American experiment in republican government. Washington relied heavily on Madison's advice, pen, and legislative skill, while Madison found Washington's prestige indispensable for achieving his goals for the new nation. Together, Stuart Leibiger argues, Washington and Madison struggled to conceptualize a political framework that would respond to the majority without violating minority rights. Stubbornly refusing to sacrifice either of these objectives, they cooperated in helping to build and implement a powerful, extremely republican constitution.
George Washington by James MacGregor Burns and Susan Dunn
All The Presidents Books
As Washington sought to interpret the Constitution's assignment of powers to the executive branch and to establish precedent for future leaders, he relied on his key advisers and looked to form consensus as the guiding principle of government. His is a legacy of a successful experiment in collective leadership, great initiatives in establishing a strong executive branch, and the formulation of innovative and lasting economic and foreign policies. James MacGregor Burns and Susan Dunn also trace the arc of Washington's increasing dissatisfaction with public life and the seeds of dissent and political parties that, ironically, grew from his insistence on consensus. In this compelling and balanced biography, Burns and Dunn give us a rich portrait of the man behind the carefully crafted mythology.
George Washington and the American Military Tradition by Don Higginbotham
In George Washington and the American Military Tradition, Don Higginbotham investigates the interplay of militiaman and professional soldier, of soldier and legislator, that shaped George Washington's military career and ultimately fostered the victory that brought independence to our nation. Higginbotham then explores the legacy of Washington's success, revealing that the crucial blending of civil and military concerns characteristic of the Revolution has been variously regarded and only seldom repeated by later generations of American soldiers.
George Washington, Man and Monument by Marcus Cunliffe
Step-by-step, Marcus Cunliffe traces the ancestral background, the childhood, the growth, the failures and achievements of George Washington. He shows readers a real person–fallible, ambitious, impatient of criticism–providing a portrait of one of America's greatest leaders.
George Washington: A Biography by John R. Alden
In George Washington, John R. Alden traces the interwoven histories of Washington and the nation he helped to create, defend, and guide toward the future. Alden recreates the major events of Washington's personal and professional life, including his boyhood in rural Virginia, his early careers as a surveyor and then a soldier in the French and Indian War, and his staid but enduring marriage. The core of the biography is devoted to Washington's leadership roles-his assumption of the post of commander in chief of the Continental Army, his part in the Constitutional Convention, and his presidency.
George Washington: A Life by Willard Sterne Randall
George Washington is the human story of a man who turned an impoverished childhood and the frequent humiliations at the hands of a mother he feared and the British generals he admired into a career of rebellion and creation. When he had worn out and nearly bankrupted his allies, Washington disbanded the victorious army he had forged, giving life to democratic government. George III once said that if Washington could give up power, he would be the greatest man of the eighteenth century. And Washington did. Twice. Here, from award-winning historian Willard Sterne Randall, is the human story of America's Founding Father.
George Washington's Mount Vernon: At Home in Revolutionary America by Robert F. Dalzell and Lee Brown Dalzell
Here we meet the planter/patriot who also genuinely loved building, a man passionately human in his desire to impress on his physical surroundings the stamp of his character and personal beliefs. As chief architect and planner of the countless changes made at Mount Vernon over the years, Washington began by imitating accepted models of fashionable taste, but as time passed he increasingly followed his own ideas. Hence, architecturally, as the authors show, Mount Vernon blends the orthodox and the innovative in surprising ways, just as the new American nation would. Equally interesting is the light the book sheds on the process of building at Mount Vernon, and on the people–slave and free–who did the work. Washington was a demanding master, and in their determination to preserve their own independence his workers often clashed with him. Yet, as the Dalzells argue, that experience played a vital role in shaping his hopes for the future of American society–hope that embraced in full measure the promise of the revolution in which he had led his fellow citizens.
George Washington's War: The Saga of the American Revolution by Robert Leckie
In the Hands of a Good Providence: Religion in the Life of George Washington by Mary Thompson
Attempts by evangelical Christians to claim Washington and other founders as their own, and scholars' ongoing attempts to contradict these claims, are nothing new. Particularly after Washington was no longer around to refute them, legends of his Baptist baptism or secret conversion to Catholicism began to proliferate. Mount Vernon researcher Mary Thompson endeavors to get beyond the current preoccupation with whether Washington and other founders were or were not evangelical Christians to ask what place religion had in their lives. Thompson follows Washington and his family over several generations, situating her inquiry in the context of new work on the place of religion in colonial and postrevolutionary Virginia and the Chesapeake.
Inventing George Washington: America's Founder, in Myth and Memory by Edward Lengel
Editor-in-Chief of the Washington Papers Project Edward G. Lengel delivers an entertaining and erudite history of America's Founding Father. In Inventing George Washington, a captivating counterpart to Lengel's General George Washington: A Military Life, the historian looks at Washington's life and writings, at the creation of his mythos, and at what his legacy means for our nation and ourselves.
Powerful Mind: The Self-Education of George Washington by Adrienne Harrison
His formal schooling abruptly cut off at age eleven, George Washington saw his boyhood dream of joining the British army evaporate and recognized that even his aspiration to rise in colonial Virginian agricultural society would be difficult. Throughout his life he faced challenges for which he lacked the academic foundations shared by his more highly educated contemporaries. Yet Washington's legacy is clearly not one of failure.
Realistic Visionary: A Portrait of George Washington by Peter R. Henriques
A focused and concise biography, Realistic Visionary re-examines the keys moments and themes of George Washington's life to present a more vivid portrait of the man himself, combining his attributes as a general, a politician as well as a citizen. Drawing on unparalleled access to Washington's own papers, letters and diaries, Professor Peter Henriques profiles the events that shaped Washington's life and career. In addition, he shows how several leading historical figures, including Washington's wife, Martha, Thomas Jefferson, Alexander Hamilton and more impacted Washington's direction in life. Tracking America's first president through his final days, Realistic Visionary tells the story of a man dedicated to public service who was able to do so while satisfying his own incredible sense of personal ambition.
The Ascent of George Washington by John Ferling
Perhaps the most revered American of all, George Washington has long been considered a stoic leader who held himself above the fray of political infighting. What has gone unnoticed about the much-researched life of Washington is that he was in fact a consummate politician, as historian John Ferling shows in this revealing and provocative new book. As leader of the Continental Army, Washington's keen political savvy enabled him not only to outwit superior British forces, but–even more challenging–to manage the fractious and intrusive Continental Congress. Despite dire setbacks early in the war, Washington deftly outmaneuvered rival generals and defused dissent from officers below him, ending the war with the status of a national icon. His carefully burnished reputation allowed Washington, as president, to lead the country under the guise of non-partisanship for almost all of his eight years in office. Washington, Ferling argues, was not only one of America's most adroit politicians, he was easily the most successful of all time–so successful, in fact, that he is no longer thought of as having been political.
The Genius of George Washington by Edmund S. Morgan
In this book Edmund S. Morgan pushes past the image to find the man. He argues that Washington's genius lay in his understanding of both military and political power. This understanding of power was unmatched by that of any of his contemporaries and showed itself at the simplest level in the ability to take command. Drawing on Washington's letters to his colleagues (many of which are included in this book), Morgan explores the particular genius of our first president and clearly demonstrates that Washington's mastery of power allowed America to win the Revolutionary War and placed the new country on the way to achieving the international and domestic power that Washington himself had sought for it.
The Grand Idea: George Washington's Potomac and the Race to the West by Joel Achenbach
The Grand Idea follows George Washington in the critical period immediately after the War of Independence. The general had great hopes for his young nation, but also grave fears. He worried that the United States was so fragmented politically and culturally that it would fall apart, and that the "West," beyond the Appalachian mountains, would become a breakaway republic. So he came up with an ambitious scheme: He would transform the Potomac River into the nation's premier commercial artery, binding East and West, bolstering domestic trade, and staving off disunion. This was no armchair notion. Washington saddled up and rode west on a 680-mile trek to the raucous frontier of America.
The Invention of George Washington by Paul K. Longmore
George Washington was the central actor in the great American drama of revolution and nation-founding. His image was an essential aspect of the role he played, but scholars have usually examined the life and the legend apart from one another. Paul Longmore here illuminates the relationship between the man, the myth, and the American people. In addition to a comprehensive look at Washington's career, we are shown how he embodied the ideals of his age and made an effort to shape contemporary perceptions of him.
The Life of Washington by Mason Locke Weems
The Political Philosophy of George Washington by Jeffry Morrison
Though Washington left little explicit writing on political philosophy, Jeffry Morrison examines his key writings, actions, education, and political and professional lives. He finds that Washington held closely to a trinity of foundational principles―classical republicanism, British liberalism, and Protestant Christianity―with greater fidelity than many of the other founding fathers. In unearthing Washington's ideological growth, Morrison reveals the intellectual heritage of his political thought and shows how these beliefs motivated him to action.
The Presidency of George Washington by Forrest McDonald
In this volume, Forrest McDonald admits that George Washington was no executive genius, and notes that a number of his advisers and cabinet members were considerably more important in formulating programs and policies than he was. Nevertheless, he maintains that, but for Washington, the office of president might not exist today. McDonald asserts that Washington's reputation as a man of integrity, dignity, candor, and republican virtue was well-deserved, and that he contributed best by serving as a symbol.
The Spiritual Journey of George Washington by Janice T. Connell
To the Farewell Address: Ideas of Early American Foreign Policy by Felix Gilbert
Washington's Farewell Address comprises various aspects of American political thinking. It reaches beyond any period limited in time and reveals the basic issue of the American attitude toward foreign policy: the tension between Idealism and Realism. Settled by men who looked for gain and by men who sought freedom, born into independence in a century of enlightened thinking and of power politics, America has wavered in her foreign policy between Idealism and Realism, and her great historical moments have occurred when both were combined. Thus the history of the Farwell Address forms only part of the wider, endless, urgent problem. Felix Gilbert analyzes the diverse intellectual trends which went into the making of the Farwell Address, and sheds light on its beginnings.
Source Article
All The Presidents Books One Through Forty-Two or Forty-Three
At Times Dull Janet's Presidential Biography Project & Blog
Best Presidential Bios The Best Presidential Biographies
Christian Science Monitor To celebrate Washington's birthday: 5 best books about George Washington
Five Books H W Brands recommends the best books on American Presidents
Huffington Post Presidents' Day History: The Must-Reads Of Presidential Biographies
Library of Congress Selected Bibliography
Mandi Lindner 44 Presidents and Their Definitive Biographies
Mashable Why I'm spending a year reading about every U.S. president
Patheos Five Great Books on George Washington
Presidential History Presidential Resources
Presidential History (Again) Pulitzer Prize Winning Books About Presidents
Presidents USA FURTHER INFORMATION ABOUT GEORGE WASHINGTON
Signature The 5 Best George Washington Biographies
The Tailored Man The 44 Best Presidential Biographies
The Washington Post The Fix's list of best presidential biographies
The Best Children's Books About Knights And Medieval History
Ranking Author David Eddings's Best Books (A Bibliography Countdown)
Ranking Author N. K. Jemisin's Best Books (A Bibliography Countdown)
The Best Books About Every US President
The Best Books To Learn About President John Adams
Ranking Author Aldous Huxley's Best Books (A Bibliography Countdown)
The Best Japanese Manga Of All-Time
The Best Ecology Books Of All-Time
Ranking Author David Sedaris's Best Books (A Bibliography Countdown)
Ranking Author Ann Patchett's Best Books (A Bibliography Countdown)
The Best Books To Read As A Freshman In High School
The Best Japanese Science Fiction & Fantasy Books
The Best Computer Science Books


text:
\section{Introduction}

The huge computation power of today's computing systems, equipped with GPUs, special ASICs, FPGAs, or multi-core CPUs, makes it possible to train deep networks using tremendous datasets. Although such high-performance systems can be used for training, actual inference in the real world may be executed on small devices such as a handheld device or an embedded controller.
Hence, various techniques (such as \cite{Hinton15} and \cite{Han16}) to achieve high prediction accuracy without increasing computation time have been studied to enable more applications to be deployed in the real world. Recently, graph compilers and optimizers, such as NNVM/TVM (\cite{Chen18}) and Glow (\cite{Rotem18}), were developed to make small and efficient executables suitable for handheld devices from trained networks.

Ensembling multiple predictions is a widely-used technique to improve the accuracy of various machine learning tasks (e.g. \cite{Hansen90}, \cite{Zhou02}) at the cost of more computation power. In the image classification tasks, for example, accuracy is significantly improved by ensembling the local predictions for multiple patches extracted from the input image to make the final prediction. Moreover, accuracy is further improved by using multiple networks trained independently to make local predictions. \cite{Krizhevsky12} averaged 10 local predictions using 10 patches extracted from the center and the 4 corners with and without horizontal flipping in their Alexnet paper. They also used up to 7 networks and averaged the prediction to get higher accuracy. GoogLeNet by \cite{Szegedy15} averaged up to 1,008 local predictions by using 144 patches and 7 networks.
In some ensemble methods, meta-learning during the training to learn how to best mix the multiple local predictions from the networks is used (e.g. \cite{Tekin16}). In the Alexnet or GoogLeNet papers, however, the significant improvements were obtained by just averaging the local predictions without the meta-learning. In this paper, we do not use meta-learning either.

Although the benefits of ensemble prediction are quite significant, one obvious drawback is its higher execution cost during inference. If we make the final prediction by ensembling 100 predictions, we need to make 100 local predictions, and hence the execution cost will be 100 times as high as that without ensembling.

\begin{figure*}[t]
\begin{center}

  \begin{tabular}{c}
    \begin{minipage}{0.5\hsize}
      \includegraphics[width=8.5cm,trim=1cm 6cm 0 0]{fig1a.pdf}
    \end{minipage}
    \begin{minipage}{0.5\hsize}
      \includegraphics[width=8.5cm,trim=1cm 6cm 0 0]{fig1b.pdf}
    \end{minipage}
  \end{tabular}
  \caption{Improvements by ensemble and probabilities of predictions in ILSVRC 2012 validation set. X-axis shows percentile of probability of first local predictions from high (left) to low (right). Ensemble reduces error rates for inputs with low probabilities but does not affect inputs with high probabilities.}

  \begin{tabular}{c}
    \begin{minipage}{0.5\hsize}
      \includegraphics[width=8cm,trim=1.8cm 7cm 0 0]{fig2a.pdf}
    \end{minipage}
    \begin{minipage}{0.5\hsize}
      \includegraphics[width=8cm,trim=1.8cm 7cm 0 0]{fig2b.pdf}
    \end{minipage}
  \end{tabular}
  \caption{Breakdown of samples into four categories based on 1) whether first local prediction is correct or incorrect and 2) whether first and second predictions are same or different. X-axis shows percentile of probability of first local predictions from high (left) to low (right). Two networks tend to make same (mis)predictions for samples predicted with a high probability (left)
  }

\end{center}
\end{figure*}

In this paper, we first describe our insights on relationship between the probability of the prediction and the effect of ensembling with current deep neural networks; ensembling does not help mispredictions for inputs predicted with a high probability, i.e. the output from the softmax, even when there is a non-negligible number of mispredicted inputs. To exploit this finding to speedup ensembling, we develop an adaptive ensemble prediction, which achieves the benefits of ensembling with much smaller additional costs. During the ensembling process, we calculate the confidence level of the probability obtained from local predictions for each input. If an input reaches a high enough confidence level, we stop ensembling and making more local predictions for this input to avoid wasting computation power. We evaluated the adaptive ensembling by using four image classification datasets: ILSVRC 2012, CIFAR-10, CIFAR-100, and SVHN. Our results showed that the adaptive ensemble prediction reduces the computation cost significantly while achieving similar accuracy to the static ensemble prediction with the fixed number of local predictions. We also showed that our statistically rigorous confidence-level-based termination condition reduces the burden of the task-dependent parameter tuning compared to the naive termination condition based on the pre-defined threshold in the probability in addition to yielding a better accuracy with the same cost (or lower cost for the same accuracy).

\section{Ensembling and Probability of Prediction}

This section describes the observations that have motivated us to develop our proposed technique: how the ensemble prediction improves the accuracy of predictions with different probabilities.

\subsection{Observations}

To show the relationship between the probability of the prediction and the effect of ensembling, we evaluate the prediction accuracy for the ILSVRC 2012 dataset with and without ensembling of two predictions made by two independently trained networks. Figure 1(a) shows the results of this experiment with GoogLeNet; the two networks follow the design of GoogLeNet and use exactly the same configurations (hence the differences come only from the random number generator). In the experiment, we 1) evaluated the 50,000 images from the validation set of the ILSVRC 2012 dataset using the first network without ensembling, 2) sorted the images by the probability of the prediction, and 3) evaluated the images with the second network and assessed the accuracy after ensembling two local predictions using the arithmetic mean. The x-axis of Figure 1(a) shows the percentile of the probability from high to low, i.e. going left (right), the first local predictions become more (less) probable. The gray dashed line shows the average probability for each percentile class. Overall, the ensemble improves accuracy well, although we only averaged two predictions. Interestingly, we can observe that the improvements only come in the right of the figure. There are almost no improvements by ensembling two predictions on the left side, i.e. input images with highly probable local predictions, even when there is a non-negligible number of mispredicted inputs. For example, in the 50- to 60-percentile range with GoogLeNet, the top-1 error rate is 29.6\% and is not improved by averaging two predictions from different networks.

For more insight into these characteristics, Figure 2(a) shows the breakdown of 5000 samples in each 10-percentile range into four categories based on 1) whether the first prediction is correct or not and 2) whether the two network makes the same prediction or different predictions. When a prediction with a high probability is made first, we can observe that another local prediction tends to make the same prediction regardless of its correctness. In the highest 10-percentile range, for instance, two independently trained networks make the same misprediction for all the 43 mispredicted samples. The two networks make different predictions only in two out of 5,000 samples even when we include the correct predictions. In the 10- to 20-percentile range, two networks generate different predictions only in three out of 139 mispredicted samples. Ensembling does not work well when local predictions tend to make the same mispredictions.

To determine whether or not this characteristic of ensembling is unique to GoogLeNet architecture, we conducted the same experiment using Alexnet as another network architecture and show the results in Figure 1(b) and 2(b). Although the prediction error rate is higher for Alexnet than for GoogLeNet, we observe similar characteristics of improvements by ensembling.

When we combine Alexnet (for the first prediction) and GoogLeNet (the second prediction), ensembling the local prediction from GoogLeNet, which yields much higher accuracy than the first prediction by Alexnet, does not give significant gain in 0- to 20-percentile range. We discuss this deeper in the supplementary material.

These characteristics of the improvements by the ensemble are not unique to an ILSVRC dataset; we have observed similar trends in other datasets.

\subsection{Why this happens}

\begin{figure}
  \begin{center}
    \includegraphics[width=8cm]{simple.pdf}
    \caption{Schematic examples of mispredictions with a high and low probabilities. A classifier may fail to learn some decision boundaries and this lead to mispredictions with high probability; ensemble cannot help them.}
  \end{center}
  \label{boundaries}
\end{figure}

Trained neural networks (or any classifiers in general) often make a prediction for a sample near the decision boundary of class A and class B with a low probability; the classifier assigns similar probabilities for both classes and a class with a higher probability (but with a small margin) is selected as the result. For such samples, ensemble works efficiently by reducing the effects of random perturbations.

While the ensemble works near the decision boundary that is properly learned by the classifier, some decision boundary can be totally missed in the trained classifier due to the insufficient expressiveness in the used model or the lack of the appropriate training data. Figure 3 shows an example of the true decision boundaries in the 2-D feature space and a classification results by a classifier which is not capable of capturing all of these boundaries. In this case, a small region of class A at the left-top is totally missed in the classification results; i.e. the samples in this region are mispredicted with high probabilities. Typically, such mispredictions cannot be avoided by ensembling predictions from another classifier trained with different random numbers since both classifiers tends to make the same mispredictions. We show this problem in more detail using an actual neural network for a simple classification task and in the supplemental material.

These results motivate us to make our adaptive ensemble prediction for reducing the additional cost of ensembling while keeping the benefit of improved accuracy. Once we obtain high enough prediction probability for an input image, doing further local predictions and ensembling will waste computation power without improving accuracy. The challenge is how to identify the condition in which to terminate ensembling. As described later, we identify the termination condition based on the confidence level of the probability works well for all the tested datasets.

\section{Related Work}

Various prediction methods that ensemble the outputs from many classifiers (e.g. neural networks) have been widely studied to achieve higher accuracy in machine learning tasks. Boosting (Freund and Schapire 1996) and Bagging (\cite{Breiman96}) are famous examples of ensemble methods. Boosting and Bagging produce enough variances in classifiers included in an ensemble by changing the training set for each classifier. Another technique to improve binary classification using multiple classifiers is Soft-cascade (e.g. \cite{Bourdev05}, \cite{Zhang08}). In Soft-cascade, multiple weak sub-classifiers are trained to reject a part of the negative inputs. Hence, as the entire strong classifier, many easy-to-reject inputs are rejected in early stages without consuming huge computation time.
Compared to Boosting, Bagging or Soft-cascade, ours is an inference-time technique and does not affect the training phase. In recent studies on image classifications with deep neural networks, random numbers (e.g. for initialization or for ordering input images) used in the training phase can give sufficient variances in networks even using the same training set for all classifiers (networks). Hence, we use networks trained using the same training set and network architecture in this study and we assume that the capabilities of local classifiers are not that different. If a classifier is too much weaker than the later classifiers as in Soft-cascade, mispredictions in the weak classifier may predicted correctly by later powerful classifiers even they are predicted with high probabilities in the local prediction by the weak classifier.

Another series of studies to accelerate classification tasks with two or few classes is based on dynamic pruning of majority voting (e.g. \cite{Lobato09}, \cite{Soto16}). Like our technique, the dynamic pruning uses a certain confidence level to prune the ensembling with the sequential voting process to avoid wasting the computation time. We show that the confidence-level-based approach is quite effective to accelerate the ensembling by averaging local predictions in many-class classifications tasks with deep neural networks when we use the output of the softmax as the probability of the local predictions. Some existing classifiers with a deep neural network (e.g. \cite{Bolukbasi2017}) take an early exit approach similar to ours. In our study, we study how the exit condition for the termination affects the execution time and the accuracy in detail and showed our confidence-level-based condition works better than the naive threshold-based conditions.

The higher execution cost of the ensembling is a known problem, so we are not the first to attack it. For example, \cite{Hinton15} also tackled the high execution cost of the ensembling. Unlike us, they trained a new smaller network by distilling the knowledge from an ensemble of networks by following \cite{Bucilua06}.

In our technique, we use the probability of the predictions to control the ensembling during the inference. Typically, the probability of the prediction generated by the softmax is used during the training of the network; the cross entropy of the probabilities is often used as the objective function of the optimization. However, using the probability for purposes other than the target of the optimization is not unique to us. For example, \cite{Hinton15} used the probabilities from the softmax while distilling the knowledge from the ensemble. As far as we know, ours is the first study focusing on the relationship between the probability of the prediction and the effect of ensembling with current deep neural networks.

\cite{Opitz99} showed an important observation related to ours. They showed that the large part of the gain of ensembling came from the ensembling of the first few local predictions. Our observation discussed in the previous section enhances Opitz's observation from a different perspective: most gain of the ensembling comes from inputs with low probabilities in the prediction.

\section{Adaptive Ensemble Prediction}

\subsection{Basic Idea}

This section details our proposed adaptive ensemble prediction method. As shown in Figure 1, the ensemble typically does not improve the accuracy of predictions if a local prediction is highly probable. Hence, we terminate ensembling without processing all $N$ local predictions on the basis of the probabilities of the predictions. We execute the following steps:

\begin{enumerate}
\item	start from $i$ = 1
\item obtain $i$-th local prediction, i.e. the probability for each class label. We denote the probability for label $L$ of $i$-th local prediction $p_{L,i}$
\item calculate the average probabilities for each class label
\begin{equation}
\left<p_L\right>_i = \frac{\sum_{j=1}^{i}p_{L,j}}{i}
\end{equation}

\item if $i < N$ and the termination condition is not satisfied, increment $i$ and repeat from step 2
\item output the class label that has the highest average probability $\mathop{\rm arg~max}\limits_{L}\left(\left<p_L\right>_i\right)$ as the final prediction.
\end{enumerate}

\subsection{Confidence-Level-Based Termination}

For the termination condition in Step 4, we propose a condition based on a confidence level.

We can use a naive condition on the basis of a pre-determined threshold $T$ to terminate the ensembling, i.e. we just compare the highest average probability $\max_{L}\left(\left<p_L\right>_i\right)$ against the threshold $T$. If the average probability exceeds the threshold, we do not execute further local predictions for ensembling. As we empirically show later, the best threshold $T$ heavily depends on task. To avoid this difficult tuning of the threshold $T$, we propose more statistically rigorous condition in this paper.

Instead of the pre-defined threshold, we can use the confidence intervals (CIs) as a termination condition. We first find the label that has the highest average probability ({\it predicted label}). Then, we calculate the CI of the probabilities using $i$ local predictions. If the calculated CI of the predicted label does not overlap with the CIs for other labels, i.e. the predicted label is the best prediction with a certain confidence level, we terminate the ensembling and output the predicted label as the final prediction.

We calculate the confidence interval for the probability of label $L$ using $i$ local predictions by
\begin{equation}
\left<p_L\right>_i \pm z \frac{1}{\sqrt{i}}\sqrt{\frac{\sum_{j=1}^{i}\left(p_{L,j}-\left<p_L\right>_i\right)^2}{i-1}}
\end{equation}
Here, $z$ is defined such that a random variable $Z$ that follows the Student's-t distribution with $i-1$ degrees of freedom satisfies the following condition: $Pr\left[Z \leq z \right] = 1 - \alpha / 2$. $\alpha$ is called the significance level and $(1 - \alpha)$ is called the confidence level. We can read the value $z$ from a precomputed table at runtime. To compute the confidence interval with small number of samples (i.e. local predictions), it is known that the Student's-t distribution is more suitable than the normal distribution. When the number of local predictions increases, the Student's-t distribution approximates the normal distribution.

We can do pair-wise comparisons between the predicted label and all other labels. However, computing CIs for all labels is costly, especially when there are many labels. To avoid excess costs of computing CIs, we compare the probability of the predicted label against the total of the probabilities of other labels. Since the total of the probabilities of all labels (including the predicted label) is 1.0 by definition, the total of the probabilities for the labels other than the predicted label are $1 - \left<p_L\right>_i$ and the CI is the same size as that of the probability of the predicted label. Hence, our termination condition is
\begin{equation}
\left<p_L\right>_i - \left(1 - \left<p_L\right>_i\right) > 2z \frac{1}{\sqrt{i}}\sqrt{\frac{\sum_{j=1}^{i}\left(p_{L,j}-\left<p_L\right>_i\right)^2}{i-1}}
\end{equation}
We avoid computing CI if $\left<p_L\right>_i < 0.5$ to avoid wasteful computation because the termination condition of equation 2 cannot be met in such cases.
Since the CI cannot be calculated with only one local prediction as is obvious from equation (3) to avoid zero divisions, we can use a hybrid of the two termination conditions. We use the static-threshold-based condition only for the first local prediction with a quite conservative threshold (99.99\% in the current implementation) to terminate ensembling only for trivial inputs as early as possible, and after the second local prediction is calculated, the confidence-level-based condition of equation (3) is used.
We also evaluated by doing pair-wise comparisons of CIs with all other labels or with the label having the second-highest probability. But the differences in the results by doing the pair-wise comparisons were mostly negligible. There can be many other criteria for the termination conditions, but our approach with the confidence level is a reasonable choice to balance multiple objectives including accuracy, computation cost and ease of tuning.

\section{Experiments}

\begin{figure}[t]
  \centering
  \includegraphics[clip,width=10cm]{fig3.pdf}
  \caption{Prediction accuracy and computation cost with static ensemble and our adaptive ensemble using different termination conditions
  Confidence-level-based condition achieved better accuracy than static-threshold-based conditions with same computation cost especially for CIFAR-10. Tuning of confidence level (CL) is less sensitive than that of static threshold.}
  \label{tradeoff}
\end{figure}

\begin{table*}[t]
  \centering
  \caption{Prediction accuracy with and without adaptive ensemble}
  \includegraphics[clip,width=11cm,trim=0 7cm 0 0]{table1.pdf}

  \centering
  \caption{Number of local predictions ensembled with and without adaptive ensemble}
  \includegraphics[clip,width=12.0cm,trim=0 13.5cm 0 0]{table3.pdf}

  \centering
  \caption{Execution time with and without adaptive ensemble}
  \includegraphics[clip,width=12.0cm,trim=0 10cm 0 0]{table2.pdf}
\end{table*}

In this section, we investigate the effects of adaptive ensemble prediction on the prediction accuracy and the execution cost using various image classification tasks: ILSVRC 2012, Street View House Numbers (SVHN), CIFAR-10, and CIFAR-100 (with fine and course labels) datasets.

For the ILSVRC 2012 dataset, we use GoogLeNet as the network architecture and train the network using the stochastic gradient descent with momentum as the optimization method. For other datasets, we use a network that has six convolutional layers with batch normalization (\cite{Ioffe15}) followed by two fully connected layers with Dropout. We used the same network architecture except for the number of neurons in the output layer. We train the network using Adam (\cite{Kingma15}) as the optimizer. For each task, we trained two networks independently. During the training, we used data augmentations by extracting a patch from a random position of the input image and using random horizontal flipping. Since adaptive ensemble prediction is an inference-time technique, it does not affect the network training.

We averaged up to 20 local predictions using ensembling. We created 10 patches from each input image by extracting from the center and the four corners with and without horizontal flipping by following Alexnet. For each patch, we made two local predictions using two networks. The patch size is 224x224 for the ILSVRC 2012 dataset and 28x28 for the other datasets. We made local predictions in the following order: (center, no flip, network1), (center, no flip, network2), (center, flipped, network1), (center, flipped, network2), (top-left, no flip, network1), ..., (bottom-right, flipped, network2). Since averaging local predictions from different networks typically yield better accuracy and hence we use this order for both our adaptive ensembling and fixed-number static ensembling.

To study the effects of our adaptive ensemble on the computation cost and accuracy, we show the relationship between them for ILSVRC 2012 and CIFAR-10 datasets in Figure \ref{tradeoff}.
In the figure, the x-axis is the number of ensembled predictions, so smaller means faster. The y-axis is the improvements in classification error rate over the baseline (no ensemble), so higher means better. We evaluated the static ensemble (averaging the fixed number of predictions) by changing the number of predictions to average and our adaptive ensemble. For the adaptive ensemble, we also evaluated with two termination conditions: with naive static threshold and with confidence interval. We tested the static-threshold-based conditions by changing the threshold $T$ and drew lines in the figure. Similarly, we evaluated the confidence-level-based condition with three confidence levels frequently used in statistical testing: 90\%, 95\% and 99\%.

From the figure, there is an obvious tradeoff between the accuracy and the computation cost. The static ensemble with 20 predictions is at one end of the tradeoff because it never terminates early. The baseline, which does not execute ensemble, is at the other end, which always terminates at the first prediction regardless of the probability. Our adaptive ensemble with the confidence-level-based condition achieved better accuracy with the same computation cost (or smaller computation cost for the same accuracy) compared to the static ensemble or the naive adaptive ensemble with a static threshold.
The gain with the confidence-level-based condition over the static-threshold-based is significant for CIFAR-10 whereas it is marginal for ILSVRC 2012. These two datasets show the largest and smallest gain with the confidence-level-based condition over the static-threshold-based condition; other datasets showed improvements between those of the two datasets shown in Figure 2.

When comparing two termination conditions in the adaptive ensemble, the confidence-level-based condition eliminates the burden of the parameter tuning compared to the naive threshold-based condition in addition to the benefit of the reduced computation cost. Obviously, how to decide the best threshold $T$ is the most important problem for the static-threshold-based condition. The threshold $T$ can be used as a knob to control the tradeoff between the accuracy and the computation cost, but the static threshold tuning is highly dependent on the dataset and task. From figure \ref{tradeoff}, for example, $T=80\%$ seems to be a reasonable choice for ILSVRC 2012, but it is a problematic choice for CIFAR-10. For the confidence-level-based condition, the confidence level also controls the tradeoff. However, the differences in the computation cost and the improvements in accuracy due to the choice of the confidence level are much less significant and less sensitive to the current task than the differences due to the static threshold. Hence task-dependent fine tuning of the confidence level is not as important as the tuning of the static threshold. The easier (or no) tuning of the parameter is an important advantage of the confidence-level-based condition.

Tables 1, 2, and 3 show how adaptive ensemble prediction affected the accuracy of predictions and the execution costs in more detail for five datasets. Here, for our adaptive ensemble, we use the confidence-level-based termination condition with a 95\% confidence level based on the results of Figure \ref{tradeoff}.
We test two different configurations: with one network (i.e. up to 10 local predictions) and two networks (up to 20 local predictions). In all datasets, the ensemble improves the accuracy in a tradeoff for the increased execution costs as expected. Using two networks doubles the number of local predictions on average (from 10 to 20) and increased both the benefit and drawback. If we use further local predictions (e.g. original GoogLeNet averaged up to 1,008 predictions), the benefit and the cost will become much more significant. Comparing our adaptive ensemble with the static ensemble, our adaptive ensemble similarly improves accuracy (from 92\% to 99\% when we use two networks and from 83\% to 99\% when we use one network) while reducing the number of local predictions used in the ensembles; the reductions are up to 6.9x and 12.7x for the one-network and two-network configurations.
The reduced numbers of local predictions result in shorter execution time; the speedup was by 2.1x to 2.8x and by 2.3x to 3.5x for the one-network and two-network configurations, respectively. The reductions in the execution time over the static ensemble are smaller than the reduction in the number of averaged predictions because of the additional overhead due to the confidence interval calculation, which is written in Python in the current implementation. Also, mini batches gradually become small as ensembling for parts of inputs terminated. The smaller batch sizes reduce the efficiency of execution on current GPUs.
Since the speedup by our adaptive technique over the static ensemble becomes larger as the number of max predictions to ensemble increases, the benefit of our adaptive technique will become more impressive if we use larger ensemble configurations.

\section{Conclusion}

In this paper, we described our adaptive ensemble prediction to reduce the computation cost of ensembling many predictions. We were motivated to develop this technique by our observation that ensembling does not improve the prediction accuracy if predictions are highly probable. Our experiments using various image classification tasks showed that our adaptive ensemble makes it possible to avoid wasting computing power without significantly sacrificing the prediction accuracy by terminating ensembles based of the probabilities of the local predictions. The benefit of our technique will become larger if we use more predictions in an ensemble. Hence, we expect our technique to make the ensemble techniques more valuable for real-world systems by reducing the total computation power required while maintaining good accuracies and throughputs.

\bibliographystyle{named}

redpajama_set_name: RedPajamaArXiv


Don't mistakenly think that only 20 videos per category are available; videos load in increments of 20 as you scroll right. Lego Toons, for example, has over 100 videos.
Developer's Channel Description: Welcome to Pumba Kids TV, your new home for kid-friendly and kid-safe content. Loaded with cartoons, cute animals, educational clips and movies kids are guaranteed to love! Channels include America's Funniest Videos, Parry Gripp, Twin Toys, Lego Toon, and We are the Davises! Whether you like toys, adorable pets, fun science experiments, or funny clips, Pumba Kids TV is the place for you!


Scott is a legend in the venue, here since the day the doors opened he is the ultimate Corn Exchange "Guru" – challenge him by finding that question he doesn't know the answer to….. impossible we say.
Since opening its doors in 1999, Neil has been a key part of the Corn Exchange team. After more than a decade heading up both the events and operational teams, Neil took over the mantle as Sales & Marketing Manager in 2013.
He is now responsible for overseeing the extensive sales and marketing activities for the business as well as managing the respective teams. Currently he is working on increasing awareness of the venue's profile locally, nationally and internationally, as well showcasing its versatility highlighting the fact that it is one of only two venues in the city that can host cabaret and dinner set ups for over 400 guests.