text:
At a southern Iraq site, unearthing the archaeological passing of time
Excavations at Lagash led by Penn's Holly Pittman have resulted in the discovery of urban neighborhoods, tens of thousands of pottery sherds, and much more.
Holly Pittman and a team of researchers studying the archaeological site of Lagash, seen at bottom center, hypothesize that the ancient city was likely a significant population center that had ready access to fertile land and people dedicated to intensive craft production. (Image: Courtesy of Lagash Archaeological Project)
When Holly Pittman and colleagues from the University of Pennsylvania and University of Pisa returned to Lagash in the fall of 2022 for a fourth season, they knew they'd find more than ceramic fragments and another kiln. With high-tech tools in hand, the team precisely located trenches to excavate a variety of features of a non-elite urban neighborhood from one of southwest Asia's earliest cities.
What surprised the researchers most was the large "tavern" they uncovered, complete with benches, a type of clay refrigerator called a "zeer," an oven, and the remains of storage vessels, many of which still contained food. "It's a public eating space dating to somewhere around 2700 BCE," says Pittman, a professor in Penn's History of Art department, curator of the Penn Museum's Near East Section, and the Lagash project director. "It's partially open air, partially kitchen area."
[Lagash] might have been something like Trenton, as in 'Trenton makes, the world takes,' a capital city but also an important industrial one. Professor Holly Pittman, Lagash project director
The find provides another glimpse into the lives of everyday people who dwelled some 5,000 years ago in this part of the world, an area Penn researchers have studied since the 1930s when the Penn Museum teamed up with Leonard Woolley and the British Museum to excavate the important archaeological site of Ur about 30 miles to the southwest. In 2019, the latest round of Lagash excavations began, and despite a short pandemic-necessitated pause, the project has real momentum, with four field seasons now complete.
To excavate most effectively, the researchers are employing cutting-edge methodologies, including drone photography and thermal imaging; magnetometry, which captures the magnetic intensity of buried features; and micro-stratigraphic sampling, a surgically precise type of excavation. To understand the city's environmental context, they've also extracted sediment cores that reflect millennia of ecological development.
"At more than 450 hectares, Lagash was one of the largest sites in southern Iraq during the 3rd millennium," Pittman says. "The site was of major political, economic, and religious importance. However, we also think that Lagash was a significant population center that had ready access to fertile land and people dedicated to intensive craft production. In that way the city might have been something like Trenton, as in 'Trenton makes, the world takes,' a capital city but also an important industrial one."
A return to Lagash
During the fourth millennium BCE, about 6,000 years ago, the mouth of the Persian Gulf lay some 150 miles farther northwest than it does today. Close to that ancient shoreline sat a trio of settlements—Girsu, Lagash, and Niĝin—which together made up the larger city-state also known as Lagash. For more than 150 years, the three cities were unified as a single powerful and wealthy political entity.
A map of southern Mesopotamia during the third millennium BCE, around 6,000 years ago. (Image: Courtesy of M. Sauvage Atlas)
Initially, archaeologists believed that the site of Girsu was Lagash. Then, in 1953, a surface inscription revealed the ancient city's true location, at a spot called Tell al-Hiba, named after the nearby village. In the 1960s and '70s, a team led by Donald Hansen of NYU's Institute of Fine Arts and Vaughn Crawford of the Metropolitan Museum of Art completed five field seasons at the ancient city focusing on monumental and administrative architecture.
The Iran-Iraq War ended that work and, save for a UCLA survey in 1984, the site remained untouched until 1990, when Hansen and colleagues returned. That group included Pittman, a former student of Hansen's who was then a newly minted associate professor at Penn. "Donald and I walked around the site and fantasized that we would start a new campaign, and it would be NYU and Penn and we would go together," Pittman recalls.
Partnering with the community
Tell al-Hiba, the site of this archaeological excavation, is also a small village. "Giving back to this community is paramount," says Holly Pittman, Lagash project director. For the primary school, the researchers helped facilitate a new courtyard, privacy wall, and bathrooms, and project manager Zaid Alrawi is working with locals he knows to identify other areas of critical need in the village.
Through its website, the team also shares important discoveries, both in English and Arabic. "Ultimately," Pittman says, "we're striving to maximize synergy and symbiosis with our hosts, as the economic and intellectual stimulus from the research creates a fundamental network of commitment, trust, and, following years of neglect, recovery."
After that single season, however, the First Gulf War halted research again. Though Pittman continued working in the Middle East for the next several decades—including analyzing and publishing the results of Hansen's Lagash fieldwork after he died—it wasn't until 2017 that she approached the State Board of Antiquities and Heritage (SBAH) in Iraq for a permit to resume work at the site.
Around that time, Zaid Alrawi had been interviewing for a postdoctoral position at the Penn Museum when he ran into Ph.D. student Reed Goodman, a former classmate now working with Pittman. "He told me about Holly and the project she wanted to resume at Lagash," Alrawi says, "that she was looking for help from someone who spoke Arabic and who knew archaeology."
Alrawi, originally from Iraq, had recently completed a Ph.D. in landscape archaeology at Penn State University; Goodman made the connection, and two weeks later, Alrawi and Pittman were on a plane. "We visited a few sites, starting with Lagash," Alrawi says. "That brought Holly a lot of emotion. We went to the collapsed dig house where she had stayed previously. It was a special time. It showed me she wasn't a random director who wanted to start a project—there was history there—and that mattered to me almost before anything else."
On the left, Pittman, director of the Lagash Archaeological Project, stands in the bedroom door of a collapsed dig house where she had stayed during previous fieldwork. On this trip, in December 2018, she and project manager Zaid Alrawi visited a few sites starting with Lagash. On the right, Pittman and Alrawi (far left) speak with local elders during the first Lagash field season in 2019. (Image: Courtesy of Lagash Archaeological Project)
Following that December 2018 visit, the rest of the team came together quickly, with Pittman at the helm and Alrawi as project manager. Augusta McMahon, then at the University of Cambridge and now at the University of Chicago, joined as field director, and Emily Hammer, an assistant professor in Penn's Department of Near Eastern Languages and Civilizations, joined as assistant field director, along with SBAH representatives and a dozen faculty and students, including Goodman. In March 2019, they had their first field season. "Nobody on the team had worked in Iraq for a long time," Pittman says. "But it was an extraordinarily successful season. We came back with a lot of information and ideas about how we wanted to go forward."
Since then, the University of Pisa's Sara Pizzimenti has become field director, taking over the role during the most recent season. Goodman has returned three times, and a handful of other Penn graduate students have participated in various aspects of the project, including supervising excavations, registering small finds, and drawing thousands of pottery sherds.
The kilns and craft production
The Lagash settlement runs about three miles from north to south and one mile across, making it one of the biggest archaeological sites in southern Mesopotamia. Just beneath its surface lies evidence that can help reconstruct the lives of everyday people during what's known as the Early Dynastic period, a 600-year stretch from 2900 to 2300 BCE, when some of the world's first cities found their form.
A portion of the site known as Area H, which sits in Lagash's central-southern section, in a zone of low, discontinuous mounding. Here, magnetometry imaging from the 2019 season, collected by Emily Hammer, and one of the 2022 seasons, collected by Paul Zimmerman, shows the magnetic intensity of buried features. (Composite image: Paul Zimmerman/Lagash Archaeological Project)
During that first season, Hammer conducted drone imagery and magnetometry that showed the site's layout. In the southern part of the site, the team noticed clear evidence of prior burning on a high rise, including an ashy surface with waste materials from pottery manufacture. Under McMahon's guidance, they opened a large, square trench, where they unearthed six ceramic kilns with oval pits made of baked mudbrick. No upper portions remained, but at the time, McMahon surmised the tops were likely domed, composed of the same mud and clay that lined them. Excavations in the spring and fall of 2022 revealed five more kilns.
To the archaeologists, the density of the kilns indicates significant ceramic production at Lagash, and, since their discovery, the structures have elicited many follow-up questions: For one, how was such craft production organized? Was it politically centralized or run by smaller, local communities? At what temperatures did the kilns operate? And what was their lifespan?
Answers to those queries alone could fill volumes, but the researchers have more, specifically about nearby architecture. Pittman says the kilns have likely survived for thousands of years—and so close to the surface, too—because they were hardened by fire. It's increasingly clear that most of the surrounding streets, alleyways, and buildings predate the kilns, with millennia of rain, flooding, and wind erasing those that would have been contemporary.
A drone photo of one of the site's trenches, which shows two rectangular pits that held clay. Such finds provide further evidence that Lagash was a site where significant craft production took place. (Image: Courtesy of Lagash Archaeological Project)
Yet recent work suggests that at least one street and workspace align, timewise, with one kiln area. "We found two rectangular pits that seem to represent stages of ceramic production," Pittman says. "One contained a pure red clay, which has ideal plastic qualities and is still used today. The other held a coarse yellow-green sand, which would serve as a tempering agent."
An open space just west of these pits, likely also used by those same ceramic makers, contained benches and a table. In another square nearby, the team excavated what seems like a domestic dwelling that had a kitchen with clay jar stoppers, bowls with food, and a grinding stone on the floor, plus a toilet several rooms over. Each discovery reveals more about the people of Lagash.
Geological coring and water
The team is making similar inroads with its grasp of Lagash's ancient environment. Scholars have hypothesized that the city was once much closer to the Persian Gulf, and as part of his doctoral research, Goodman is working to determine when and why that distance changed. "I've been looking at the relationship between urbanization in southern Iraq and changing waterscapes," Goodman explains. "That's how I fell in with Holly."
On the left, doctoral student Reed Goodman (left) uses a handheld tool called an auger while being filmed for a documentary. During the 2019 season, he collected about 40 feet of sediment core—like the ones seen in the middle image—from 12 locations. During the Spring 2022 field season, the Penn researchers worked with local drill technicians to bore some 80 feet into the earth. (Image: Courtesy of Lagash Archaeological Project)
In 2019, he conducted the first geoarchaeological studies at Lagash, using a handheld device called an auger to collect about 40 feet of sediment from 12 locations. More recently, the project teamed up with local drilling technicians to reach about 80 feet down. Getting to such depths has provided tens of thousands of years of environmental proxy information, covering the late Pleistocene, which began around 2.5 million years ago and ended approximately 11,700 years ago, and the Holocene, which directly followed the Pleistocene.
Goodman is studying the samples with Liviu Giosan of the National Ocean Sciences Accelerator Mass Spectrometry facility at Woods Hole Oceanographic Institution. This work includes measuring a sample's elemental composition with X-ray fluorescence, as well as its carbon isotopic signatures to differentiate downstream sediment sources, plus investigating ancient organisms like foraminifera and ostracoda, which reflect fresh, brackish, and marine conditions.
"In the Pleistocene, southern Iraq was very different than today, with fast-flowing and deeply incised rivers," says Goodman, who is in his final year of a Ph.D. in Penn's Art and Archaeology of the Mediterranean World graduate group. Goodman and colleagues are finding that by Lagash's heyday, as the heart of the Tigris-Euphrates delta migrated southeast, the climate shifted from wetter to drier, posing an increasing challenge for the local population.
Combining all this information, "we'll see deltaic evolution in this ecological cauldron across centuries," Pittman says. "This is something people have theoretically modeled, but we're getting actual dates of water regime changes, which will be revolutionary. It's environmental science at the cutting edge."
A picture begins to emerge
Each discovery reveals another layer of the landscape and more about the people of ancient Lagash, enabled by what Alrawi calls a "modern" excavation approach. "It's not like old-time archaeology in Iraq," he says. "We're not going after big mounds expecting to find an old temple. We use our techniques and then, based on scientific priority, go after what we think will yield important information to close knowledge gaps in the field."
Craft production is one such example; so is knowing the proximity to the sea. But there are many others. "As you excavate, you analyze and create a story that we hope gets closer and closer to the reality of the past," Alrawi says.
To improve climate models, an international team turns to archaeological data
A quest to restore cultural heritage in Iraq, one site at a time
Penn team expands cultural heritage work in Iraq, backed by new funding
The story the bowls tell
From 2,800-year-old charred food lumps, a window into past civilizations
For her part, Pittman says she's thrilled at the progress of this now well-oiled machine. "We're all completely committed to the collective success of this project," she says. "Iraq has suffered so profoundly for many years. I hope it will find stability and be able to enjoy and nurture its great and important antiquity. Our role is to help make that happen."
That played out this past fall as the team uncovered the tavern. Rather than digging according to architectural construction phases, the Lagash Archaeological Project is using an approach championed by Pisa's Pizzimenti, who excavates by microstratigraphic layers, thin lens by thin lens horizontally, across a wide swath, "like doing very careful surgery," Pittman says. "Just 50 centimeters down, we were able to capture all of this. We were happily astounded."
Pittman and her colleagues feel such a sentiment time and again at Lagash, an ancient settlement that was part of a network of ancient cities in Iraq, individual sites connected if not by physical proximity, then by economics and politics, industry and natural resources. A variety of artifacts there show the archaeological passing of time, and despite all that's already been exposed or imaged, there's still so much yet to learn.
Homepage image: A drone photo of the trenches excavated in Fall 2022, the most recent fieldwork season. The closest trench shows the tavern with a type of clay refrigerator called a "zeer," an oven, and benches. (Image: Courtesy of Lagash Archaeological Project)
Holly Pittman is Lagash project director, as well as a professor in the History of Art department in the School of Arts & Sciences and curator of the Near East Section at the University of Pennsylvania Museum of Archaeology and Anthropology.
Emily Hammer is an assistant professor in the Near Eastern Languages and Civilizations Department and the Price Lab for Digital Humanities in Penn's School of Arts & Sciences.
Zaid Alrawi is project manager for the Lagash and Ur projects at the Penn Museum.
Reed Goodman is a seventh-year doctoral student in the Art and Archaeology of the Mediterranean World graduate group in the School of Arts & Sciences at Penn.
Dozens of other researchers and graduate students have conducted work in Lagash during field seasons in 2019, 2021, and two in 2022. In addition to Pittman, Alrawi, and Goodman, the most recent trip included Sara Pizzimenti from the University of Pisa; Penn graduate students Katherine Burge, Kristina Donnally, and David Mulder; Pisa graduate students Sara Quaggio, Salomè Zaffarana, and Gaia Zuccoli; Clélia Paladre of the Louvre; field archaeologist and technologist Paul Zimmerman, a Ph.D. from Penn Anthropology; writer and archaeologist Linda Jacobs; and Emanuele Taccola, a GIS specialist at the University of Pisa. For more information about the Lagash Archaeological Project, visit web.sas.upenn.edu/lagash.
Michele W. Berger Writer
Anthropology,
Archaeology,
Art History,
Penn Museum,


Give a try to one of these bohemian chic dresses. Every hot season comes with creative and unique looks. I am a big fan of boho frocks. Why? They are extremely comfy, easy to wear and versatile. Each one of these designs can be completed with bright necklaces, bracelets, hoop earrings, etc. This year's runway was full of bold boho dresses, that's why I highly recommend to take a close up look at these pretty creations. In this street style collection are gathered flowy and lightweight dresses completed in bright prints and uber chic details. If you want to take things to the next level, add cool accessories, like fedoras, shawls, eye-catching bags, belts and other accessories. There are numerous of great boho dress styles to choose from, all you need is to follow this trend in your own unique way. Hope you gonna love these styles, just scroll down to find out more.
Printed long fringed kimono is worn atop paisley dress. Complete this look by adding cuffed glossy black pumps. I am so in love with drop earrings and printed structured bag.


text:
In Solidarity With Haiti
Unfathomable devastation, destruction and death.
That is what we all watched, in profound sorrow and disbelief, as events unfolded following the January 12, 2010, earthquake in Haiti. We learned of the death of the Archbishop of Port-au-Prince, Joseph Miot, who was found under the rubble of his residence, according to one news report. There were other equally tragic news bulletins - seminarians trapped in the rubble of the destroyed seminary; priests there for a retreat, killed in the quake; church buildings - the seminary, big churches, the cathedral - reduced to rubble.
The Pontifical Mission Societies in the United States have established a special fund for long-term assistance for the Church in Haiti, as it will continue to respond in the wake of this natural disaster - the Haitian Solidarity Fund.
We applaud and encourage support of the efforts of social service and relief agencies, such as Catholic Relief Services, who are responding to the immediate and overwhelming humanitarian needs of our suffering brothers and sisters in that part of the world.
As the Church in Haiti assesses the massive damages and the overwhelming needs, they will need financial assistance as they help to rebuild the infrastructure of the Church there that will, in turn, be critical to serve the poor and enable the suffering here to rebuild their lives. And thus, the need for this fund.
We encourage the generous hearts of Catholics in the United States as they reach out in prayer and in sacrifice to the men, women and children who suffer even more now in this country. I ask, above all, for prayers for the Church there, and for missionaries, who remain witnesses of the great love and compassion of our Lord, and of His sustaining hope.
To support our Haitian Solidarity Fund - for longterm assistance - please follow this link, and select that option on the page. You may also send a contribution for that Fund to our national office at 70 West 36th Street, 8th Floor, New York, NY 10018. To offer help to Catholic Relief Services, for immediate relief and humanitarian assistance, please go to their website.
You may also visit our Facebook page to post thoughts, prayers and photos related perhaps to your own mission experiences in Haiti; let our sharing be a source of strength for and solidarity with all who are suffering.

redpajama_set_name: RedPajamaCommonCrawl


We had a lovely trip to the zoo to check out the Halloween enrichment extravaganza. The animals got pumpkins filled with tasty treats. It was really great watching them the River Hogs and Sun Bears are among our most favourite. We also checked out the new tiger enclosure which is really cool. The tiger even walked over the tunnel while we were in it. The only downside was that it took ages to sort through the hundreds of photos. Ah well, all done now. More pics on flickr as always.


So, Waistcoat Wednesday us upon us. In tribute to Gareth Southgate, and to the relief of those who decided to bet long on waistcoats when they fell out of fashion approximately a million years ago, the British public is dressing like the England manager. From TV presenters all the way down to your office's resident "character", there are so many people doing it that Marks and Spencer, the makers of the waistcoats, have run out of them.
I am one of those impressionable idiots, which means that I am now able to tell you exactly what you learn when you wear Gareth Southgate's waistcoat for a day.
Gareth Southgate is tall enough to gracefully pluck things from the highest supermarket shelf, yet slim enough to fit what looks like a 30" waist. He is a former professional athlete and by the look of his arms I am guessing he works out most days. I am not a former professional athlete and I do not work out most days. I reckon I am just the right side of looking like the Fat Controller, but I am not certain. What about you? Only you can decide. Choose wisely.
Nothing, not even a waistcoat that at times feels as thick as an oven glove, is strong enough to protect you from the jibes of your friends and colleagues. "Flight attendant," they call you. "Cruise conjuror. Undertaker." It is true that you look ridiculous. There is nothing you can do. Truly this is harder to endure than missing a crucial penalty against Germany.
It's summer! It's hot! Yet this suit is dark and made of wool! If you left a dog in this suit for half an hour you would probably get prosecuted by the RSPCA.
It's rare that I am grateful for having filled my adolescent brain purely with football statistics – didn't fancy picking up a book, huh, Past Me? Preferred reading Shoot Monthly, did you? Yeah? – but today is one of those occasions. If you are wearing a waistcoat but don't know Gareth's backstory then you are going to get found out.
I won't bore you with the debate, but be warned. I will simply imitate Gareth, who started with the button done up, and but in England's last game had it undone, presumably to pacify an elderly relative.
6. Apparently… apparently this is for charity?
Huh? Oh! It says here that Waistcoat Wednesday was instigated by Bloodwise, a blood cancer charity, and that you can text WAISTCOAT to 70800 to donate £5. Now I know what those little pockets are for: spare change.
Ugh. Thanks for ruining it, Piers.


text:
Innovative Meetings
Why More Medical Meetings Are Looking at Airport Hotels
With time-strapped health-care professionals as their audience, more organizers are making airports their final event destination.
by David McMillin
While many attendees tack on vacation time to business events, that's a luxury others don't have time for. Not all prospective attendees are looking to immerse themselves in all a destination has to offer — especially health-care professionals.
As physicians deal with new regulations and overwhelming patient loads, they are paying particular attention to the time required to get from point A to B in order to earn CMEs. According to research conducted by global health-care firm Ashfield, physicians are only willing to spend 4.7 hours traveling to a meeting — a decrease of more than two hours compared with the same study in 2016. That heightened demand for a more efficient travel experience is leading some event organizers to reconsider downtown locations in favor of venues that lie just beyond the runway.
"More medical meetings are looking at airport properties," Pat Schaumann, senior director of health-care compliance at Maritz, told Convene. "It's easy to get in and out, which is a big selling point for physicians who don't want to be away from their practices for a long time."
Schaumann added that regulatory requirements typically limit medical professionals to no higher than four-star properties. Since airport hotels skew away from the five-star, luxury accommodations category, organizers can easily check that box. From May 17–18, non-profit addiction-recovery organization Aliviane welcomed more than 200 participants to the Radisson Hotel at the El Paso, Texas Airport. While a large portion of the audience was within driving distance of the property, Susie Villalobos, regional evaluator at Aliviane, told Convene that the location made a difference for some of the health-care professionals.
"We had a handful of guests who needed to travel [via air] to the conference," Villalobos said. "Those who did appreciated the closeness of the airport location, which was less than five minutes away."
Villalobos said that Aliviane will be looking to host events at airport properties in the future. She's not alone. Here is a sampling of health-care events held at airport hotels within the last month:
From Sept. 14–15, attendees earned up to 11 CME credits at the American Association for the Study of Liver Diseases' Hepatic Fibrosis Single Topic Conference at the Hyatt Regency DFW International Airport.
The World Congress on General Pediatrics & Adolescent Medicine welcomed medical professionals to the Doubletree by Hilton Hotel at the Philadelphia Airport From Sept. 17–18.
The increase in airport-venue popularity is not confined to the U.S. From Oct. 2–3, participants at the MENA Health Insurance Congress are scheduled to meet at the Millennium Airport Hotel in Dubai for the MENA Health Insurance Congress.
Benefits and Considerations
Skipping the second leg of the journey once they land in a destination certainly has its advantages for attendees, but organizers also see the perks of holding events at airport venues. Sally Ableitner, director of sales and marketing at the InterContinental Minneapolis – St. Paul Airport, said that the property's largest clients are companies in the medical-device industry. Since the hotel opened this past July, Ableitner estimates that it has hosted at least 50 events each month for pharmaceutical and medical-device companies. Those events range from intimate five-person gatherings to bigger, 200-person meetings. While the size varies, the questions that all organizers ask before booking are typically similar.
"When prospective clients call, they're most interested in understanding how we are connected to the airport," Ableitner said. "They're focused on the transfer time. The smaller amount of transfer from the gate, the more time they can have one-on-one with their team members. They want to maximize opportunities to collaborate."
There's another key benefit: cost. "Organizers are happy to avoid paying for transportation," Ableitner said. "Instead of paying for 10 individuals to take car services to a corporate office, one representative from that office hosts the meeting at the hotel. Compared with one person per vehicle [transit to other venues], the savings can be astronomical."
Organizers will soon be able to add up those savings in more destinations. Hotels with direct airport access are slated to open in Atlanta, San Francisco, New York (JFK), and Dublin within the next two years. Those properties will stand out on the map for some medical event organizers, but Schaumann pointed out that it's unlikely any of these hotels will welcome large annual gatherings. "You won't see many large-scale medical meetings at airport hotels," Schaumann said. "They tend to be sites for smaller events like advisory board gatherings."
And while the Ashfield study spotlighted health-care practitioners' desire to spend less time traveling to a conference compared to the 2016 survey, the opportunity to visit a new city or location was a slightly more important contributing factor in their decision-making criteria than two years ago. "Destinations are clearly still a consideration as to whether or not a health-care professional is compelled to attend," the report concluded. "The decision and appropriate venue selection remains important, despite the increasingly vigorous regulatory environment."
When the appropriate venue is an airport hotel, they can make the transition from airline passenger to session attendee in less time. But that doesn't mean they want to passively collect their CME hours and be on their way home. Eight out of 10 respondents to the Ashfield study said meetings should have more interactive activities like workshops, and 82 percent are looking for active panel discussion between speakers and participants.
Ashfield's Science of Healthcare Professional Meetings 2018 is a study of 237 respondents in the U.S., U.K., France, and Germany, practicing in four medical specialties: oncology, neurology, diabetes/endocrinology, and cardiology. Their responses indicate that meeting attendance remains a key component of health-care professionals' education and development: Two-thirds of respondents said that access to networking opportunities with global experts and key opinion leaders is their primary motivator for attendance.
Yet more than half said that increasing regulatory barriers continue to restrict their ability to attend face-to-face conferences. And time is also a concern. U.S. respondents said the ideal number of days spent away from their practice is 2.5 per meeting, while the French respondents prefer 1.9 days and Germans 1.7 days. Additionally, 57 percent of healthcare respondents said meetings should allocate less time for non-educational activities.
Download Ashfield's white paper on health-care professional events at convn.org/ashfield.
Filed Under: Departments, Innovative Meetings, Magazine
David McMillin
David McMillin is staff writer at PCMA.


text:
using System;
using Newtonsoft.Json;
using VkNet.Utils;

namespace VkNet.Model;

/// <summary>
/// Обложка чата
/// </summary>
[Serializable]
public class ChatPhoto
{
	/// <summary>
	/// URL копии фотографии с шириной 50 px
	/// </summary>
	[JsonProperty(propertyName: "photo_50")]
	public string Photo50 { get; set; }

	/// <summary>
	/// URL копии фотографии с шириной 100 px
	/// </summary>
	[JsonProperty(propertyName: "photo_100")]
	public string Photo100 { get; set; }

	/// <summary>
	/// URL копии фотографии с шириной 200 px
	/// </summary>
	[JsonProperty(propertyName: "photo_200")]
	public string Photo200 { get; set; }

	#region Методы

	/// <summary>
	/// Разобрать из json.
	/// </summary>
	/// <param name="response"> Ответ сервера. </param>
	/// <returns> </returns>
	public static ChatPhoto FromJson(VkResponse response) => new()
	{
		Photo50 = response[key: "photo_50"],
		Photo100 = response[key: "photo_100"],
		Photo200 = response[key: "photo_200"]
	};

	#endregion
}

redpajama_set_name: RedPajamaGithub


Wonderful holiday home in the South Coast of Dorset. All accommodation is one ground floor level with direct private access to Ringstead beach, so is perfect for those who with to wake up and in two minutes be on the beach. Ringstead beach is all-year dog-friendly, so bring two of your furry friends along for a seaside holiday. Open plan living area with sitting and dining were decorated nicely and is perfect for families or friends to relax together around the fire and share day's adventures.
A separate and modern kitchen is well-equipped and is looking out to a very well-maintained garden. All bedrooms have a coastal theme running though all property which gives that seaside holiday feeling. The location is popular for sunrises and sunsets and it's partially owned by the National Trust as it's unspoilt, very well sought after. The coastal path stretches from Ringstead to both sides.
Venture East and you will reach gorgeous arc of Durdle Door, Lulworth cove, Kimmeridge bay and Chapman's pool.
Head West and find seaside town of Weymouth, the Isle of Portland and Abbotsbury, West Bay. A seasonal shop is not too far from the property where you can purchase daily essentials.
The nearest bigger town is Weymouth with supermarkets and bigger shops. The Creek is a wonderful base to explore the Dorset coast's main attractions and enjoy a seaside holiday with all family or friends.
On one floor, Two bedrooms: 1 x king-size, 1 x twin room, third bed is built in single bed in corridor (suitable for a child). Bathroom with bath, shower over. Separate WC with basin. Kitchen. Utility room. Sitting room with TV / Diner.
Ringstead Bay is an unspoilt pebble beach on the Jurassic coastline surrounded by striking cliffs and farmland. The original village mentioned in the doomsday book no longer exists, but there is now a small vibrant community nestled at the back of the bay in a variety of unique houses. You can enjoy walking the South West Coast path or meander through the woodland walks, whatever your preference this is a gorgeous place to be outside. When you are ready to explore further afield, you will find Lulworth Cove and Durdle door a short 11 miles away and Weymouth with it's beautiful beach and plentiful restaurants and cafes 7 miles to the west. The Bovington Tank Museum, Monkey World Rescue, and Tyneham Village all offer varied and interesting days out just a short drive from Ringstead, presuming you can tear yourselves away from the unspoilt beach.
Note: Communal Washing and drying facilities. Note: No Wi-Fi. Note: Single bed in corridor (Suitable for a child). Note: Garden is not enclosed, with open access to Ringstead beach, supervision advised for children.


Paul Pogba is having the best season of his career.
Pogba bagged a brace against West Ham as United secured all three Premier League points.
The Frenchman netted a penalty in both halves. He ditched his long run-up and went for a more conventional approach, converting both penalties in tidy fashion.
His statistics from Saturday's game makes for great reading.
Pogba now has an impressive 13 league goals to his name not to mention nine assists. In all competitions, he's scored 16 goals.
And since Ole Gunnar Solskjaer took over at Old Trafford, Pogba has starred for the Reds.
From scoring the most goals to the most assists.
Meanwhile, United move above Arsenal to fifth place with 64 points on the board.
United resume their Champions League campaign by facing Barcelona in the return leg on Tuesday. They trail by one-goal ahead of the second leg at the Camp Nou.


Get an alert with the newest ads for "bengal" in Toronto (GTA).
Gently used Reebok Jersey of Chad "Ochocinco" Johnson. Name and numbers are stitched on the Jersey. Pick up is at Oakwood and Vaughn.
1999 Playmates Toys Approx 6" high. Brand New in Box. Never Opened Can ship anywhere in Canada by Expresspost. Payment by etransfer or Paypal. Send me your postal code for shipping rates.
Cat Exercise Wheel ( Bengals). Professionally made, solid wood.
Exercise wheel for cats, mostly Bengals. Professionally made. Solid wood. My Bengal used it every day until she passed away. About two feet in diameter. Exact measurements to be posted shortly..
7$ each I have galaxy gray, witching hour matte, bengal tiger, 081 orange, truly everlasting, fuchsia flicker, sweet & sour, tokyo tizzy.


text:
Non-Exhaustive List of Trademarks
The trademarks, service marks, and trade dress of Rubicon Communications LLC d/b/a Netgate ("Netgate") and Electric Sheep Fencing LLC ("ESF") may not be used without the prior express written permission of an officer or director of Netgate or ESF (a) in connection with any product or service that is not Netgate or ESF's, (b) in any manner that is likely to cause confusion among customers, (c) in such a manner that may weaken or dilute Netgate or ESF's rights in the trademarks, service marks or trade dress, or the goodwill therein, or (d) in any manner that disparages or discredits Netgate or ESF.
Netgate and ESF also own domestic and international rights to the trademarks, service marks, and trade dress depicted by or in the graphics, logos, page headers, button icons, scripts, and service names included in or made available through Netgate and ESF product and service offerings.
All other trademarks or service marks not owned by Netgate or ESF that may appear in any products or services offered are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Netgate or ESF.
ESF Marks -
ESF marks include but are not limited to the following:
PFSENSE® (U.S. Reg. Nos. 3571276 and 4414546)
PFSENSE® (International Reg. No. 1176766)
Netgate® Marks -
Netgate marks include but are not limited to the following:
NETGATE® (U.S. Reg. Nos. 3161285, 3146119 and 5319789)
TNSR® (U.S. Reg. No. 5942018)
Trademark Usage Guidelines for Third-Party Deliverables
Intellectual property rights, including trademarks and service marks, and the goodwill represented thereby, are important corporate assets. To help maintain these rights, it is necessary to properly use the Netgate brand. The following guidelines help protect those rights by ensuring consistent use of terminology.
Using Trademarks Properly
Netgate and ESF require that you use trademarks properly in the following manner:
Remember that a trademark or service mark, when properly used, functions as an adjective in a sentence. Therefore, appropriate use of a mark in a sentence will be followed by a noun or generic descriptor, such as "Netgate® software," "pfSense® software, or "Netgate® firewall."
Do not use the mark in plural or possessive forms (Netgates or "Netgate's").
Do not separate the mark (Net Gate or PF Sense).
Use the registered mark symbol ® next to the last "E" in Netgate and next to the last "E" in pfSense on the most prominent usage per page (such as in banners, captions, headers, etc.).
If you have any questions regarding trademark usage not covered in this guide, contact legal@netgate.com.
Rules for Trademark Symbol Usage
The following sections explain the requirements for using legends that provide legal protection for intellectual property assets. Netgate and ESF rely upon U.S. and foreign registrations and common law (unregistered) rights to protect its trademarks. Appropriate symbols should be used to indicate the status of its trademarks.
In the case of registered marks, you should acknowledge them and indicate their status by including a registered trademark symbol (®) after the first instance of the trademark within the most prominent usage on a document or on the most prominent usage per page (in materials such as websites).
For trademarks that are not registered, use the common law trademark symbol (™) for products, or "SM" for services, after the first and most prominent instance of the mark's use within a document or on each page of a website.
Trademark Status
As of the date of these guidelines, you should use the appropriate symbols with the following marks:
Netgate®
pfSense®
TNSR®
The Netgate "N" symbol:
The Netgate "N" symbol locked up with the Netgate wordmark (the Netgate logo):
Company Name Usage
The full legal company names are "Rubicon Communications LLC" and "Electric Sheep Fencing LLC." The NETGATE and PFSENSE marks are not interchangeable with the company names. Proper attribution of rights is as follows (if use has been authorized in writing, then please contact legal@netgate.com for proper language reflecting such authorization):
pfSense® is a registered mark owned by Electric Sheep Fencing LLC ("ESF") and may not be used without the prior express written permission of ESF.
Netgate® is a registered mark owned by Rubicon Communications LLC and may not be used without the prior express written permission of Rubicon Communications LLC.


text:
Dave Bartkowiak Jr.
Tags: News, Detroit, Wayne County
3 people shot during celebration of woman's life after funeral on Detroit's west side
Shooting erupts outside house at Lauder and Belton streets
Three people were shot after a funeral June 5, 2017. More than 100 people had gathered for a celebration of life outside a home at Lauder and Belton streets on Detroit's west side. (WDIV)
DETROIT – What was supposed to be a celebration of a woman's life turned into a violent shooting Monday night outside a home near Lauder and Belton streets on Detroit's west side.
Detroit police said people were gathering outside the home after a funeral. About 150-200 people were celebrating the woman's life when an argument erupted. Shots were fired.
Three people were struck by the bullets. One man was shot in his leg. Another man was shot in the torso. A third person also was shot, but it's unclear where. All three shooting victims suffered non-life-threatening injuries.
Here is the Local 4 report from Monday night:
Dave Bartkowiak Jr. is the digital managing editor for ClickOnDetroit.


All around the world this day will be a chance to stand in solidarity with trans people in prison. Join us for letter writing nights, film screenings, workshops, forums, vigils, actions, fundraisers and parties. Check the EVENTS page to see what's happening near you. Or we can support you to host your own event.
This grassroots project, lead by trans prisoners and their supporters around the world, is an annual event to give solidarity and support for those behind bars, amplify the voices of trans prisoners and to organise with others both inside and outside. For more information click HERE.


text:
<!DOCTYPE HTML>

	<meta charset="utf-8">
	<meta name="viewport" content="width=device-width, initial-scale=1">

	<title>Twilio Contact Center: Live Chat</title>

	<link rel="stylesheet" href="/styles/bootstrap.min.css" type="text/css" />
	<link rel="stylesheet" href="/styles/default.css" type="text/css" />
	<link href="//maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css"
	type="text/css" rel="stylesheet" />

Twilio Contact Center: Live ChatBack Home

Live ChatBack Home

					<form name="supportChatForm" class="well col-md-12" ng-hide="session.isInitialized">

			           <input ng-model="user.identity" name="identity" ng-minlength="3" ng-maxlength="20" ng-required="true" type="text" class="form-control" ng-disabled="session.isLoading" placeholder="Your Name">

			          <button type="submit" ng-disabled="session.isLoading || supportChatForm.$invalid || supportChatForm.$pristine" class="initiate btn btn-primary" ng-click="initChat()" >Go</button>

Name is too short

Name is too long

Name is required

					</form>

Session expired, please reload page

{{message.body}}

{{message.author}}

 {{message.timestamp | time}}

{{typingNotification}}

						<form class="chat-message-form form-horizontal" name="chatMessageForm" ng-hide="session.expired" style="margin-top:5px">

									<input ng-model="message" ng-model-options="{ debounce: 250 }" ng-required="true" autocomplete="off" ng-focus="focused()" name="text" type="text" class="form-control" placeholder="say something">

									<button ng-disabled="chatMessageForm.$invalid || chatMessageForm.$pristine" class="btn btn-primary" ng-click="send()">Send</button>

						</form>

Call us:

{{configuration.twilio.callerId}}

Request a callback?

				click here


Allow Mr Mack to record those special memories, and shared family moments.
Our action DVD's bring the memories to you. Banquets, Senior gifts, and school archives.
The Catholic High School sports league offers a wide variety of sport options for high school aged students.
Does your family own a Mr Mack Photograph?


text:
\section{Introduction}

\begin{figure}
\centering
\includegraphics[width=1.0\linewidth]{fig/intro}
\caption{Real-world large-scale datasets often display the phenomenon of long-tailed distributions. The extreme imbalance causes tremendous challenges on the classification accuracy, especially for the tail classes. Class re-balancing strategies can yield better classification accuracy for long-tailed problems. In this paper, we reveal that the mechanism of these strategies is to significantly promote classifier learning but will unexpectedly damage the representative ability of the learned deep features to some extent. As conceptually demonstrated, after re-balancing, the decision boundary ({\em i.e.}, black solid arc) tends to accurately classify the tail data ({\em i.e.}, red squares). However, the intra-class distribution of each class becomes more separable. Quantitative results are presented in Figure~\ref{fig:representation and classifier learning}, and more analyses can be found in the supplementary materials.}
\label{fig:introduction}
\end{figure}

\begin{figure*}[t]
	\centering
	\begin{minipage}[t]{0.48\textwidth}
		\centering
		\includegraphics[width=6cm]{fig/fusion-matrix-cifar100}
	\end{minipage}
	\begin{minipage}[t]{0.48\textwidth}
		\centering
		\includegraphics[width=6cm]{fig/fusion-matrix-cifar10}
	\end{minipage}
	\caption{Top-1 error rates of different manners for representation learning and classifier learning on two long-tailed datasets CIFAR-100-IR50 and CIFAR-10-IR50~\cite{ldam}. ``CE'' (Cross-Entropy), ``RW'' (Re-Weighting) and ``RS'' (Re-Sampling) are the conducted learning manners. As observed, when fixing the representation (comparing error rates of three blocks in the vertical direction), the error rates of classifiers trained with RW/RS are reasonably lower than CE. While, when fixing the classifier (comparing error rates in the horizontal direction), the representations trained with CE surprisingly get lower error rates than those with RW/RS. Experimental details can be found in Section~\ref{sec:how class re-balancing work}.}
	\label{fig:representation and classifier learning}
\end{figure*}

With the advent of research on deep Convolutional Neural Networks (CNNs), the performance of image classification has witnessed incredible progress. The success is undoubtedly inseparable to available and high-quality large-scale datasets, {\em e.g.}, ImageNet ILSVRC 2012~\cite{imagenet}, MS COCO~\cite{coco} and Places Database~\cite{zhou2017places}, \etc. In contrast with these visual recognition datasets exhibiting roughly uniform distributions of class labels, real-world datasets always have skewed distributions with \emph{a long tail}~\cite{kendall1948advanced,van2017devil}, {\em i.e.}, a few classes (a.k.a. \emph{head class}) occupy most of the data, while most classes (a.k.a. \emph{tail class}) have rarely few samples, cf.~Figure~\ref{fig:introduction}. Moreover, more and more long-tailed datasets reflecting the realistic challenges are constructed and released by the computer vision community in very recent years, {\em e.g.}, iNaturalist~\cite{cui2018large}, LVIS~\cite{Gupta2019LVIS} and RPC~\cite{wei2019rpc}. When dealing with such visual data, deep learning methods are not feasible to achieve outstanding recognition accuracy due to both the data-hungry limitation of deep models and also the extreme class imbalance trouble of long-tailed data distributions.

In the literature, the prominent and effective methods for handling long-tailed problems are class re-balancing strategies, which are proposed to alleviate the extreme imbalance of the training data. Generally, class re-balancing methods are roughly categorized into two groups, {\em i.e.}, re-sampling~\cite{shen2016relay,buda2018systematic,japkowicz2002class,buda2018systematic,he2009learning,byrd2019effect,drummond2003c4,more2016survey,chawla2002smote} and cost-sensitive re-weighting~\cite{huang2016learning,wang2017learning,cb-focal,ren18l2rw}. These methods can adjust the network training, by re-sampling the examples or re-weighting the losses of examples within mini-batches, which is in expectation closer to the test distributions. Thus, class re-balancing is effective to directly influence the classifier weights' updating of deep networks, {\em i.e.}, promoting the classifier learning. That is the reason why re-balancing could achieve satisfactory recognition accuracy on long-tailed data.

\begin{figure*}[t]
	\centering

	\includegraphics[width=0.7\linewidth]{fig/framework_revised}

	\caption{Framework of our Bilateral-Branch Network (BBN). It consists of three key components: 1) The \emph{conventional learning branch} takes input data from a uniform sampler, which is responsible for learning universal patterns of original distributions. While, 2) the \emph{re-balancing branch} takes inputs from a reversed sampler and is designed for modeling the tail data. The output feature vectors $\bm{f}_c$ and $\bm{f}_r$ of two branches are aggregated by 3) our \emph{cumulative learning strategy} for computing training losses. ``\emph{GAP}'' is short for global average pooling.}
	\label{fig:structure
\end{figure*}

However, although re-balancing methods have good eventual predictions, we argue that these methods still have adverse effects, {\em i.e.}, they will also unexpectedly damage the representative ability of the learned deep features ({\em i.e.}, the representation learning) to some extent. In concretely, re-sampling has the risks of over-fitting the tail data (by over-sampling) and also the risk of under-fitting the whole data distribution (by under-sampling), when data imbalance is extreme. For re-weighting, it will distort the original distributions by directly changing or even inverting the data presenting frequency.

As a preliminary of our work, by conducting validation experiments, we justify our aforementioned argumentations. Specifically, to figure out how re-balancing strategies work, we divide the training process of deep networks into two stages, {\em i.e.}, to separately conduct the representation learning and the classifier learning. At the former stage for representation learning, we employ plain training (conventional cross-entropy), re-weighting and re-sampling as three learning manners to obtain their corresponding learned representations. Then, at the latter stage for classifier learning, we first fix the parameters of representation learning ({\em i.e.}, backbone layers) converged at the former stage and then retrain the classifiers of these networks ({\em i.e.}, fully-connected layers) \emph{from scratch}, also with the three aforementioned learning manners. In Figure~\ref{fig:representation and classifier learning}, the prediction error rates on two benchmark long-tailed datasets~\cite{ldam}, {\em i.e.}, CIFAR-100-IR50 and CIFAR-10-IR50, are reported. Obviously, when fixing the representation learning manner, re-balancing methods reasonably achieve lower error rates, indicating they can promote classifier learning. On the other side, by fixing the classifier learning manner, plain training on original imbalanced data can bring better results according to its better features. Also, the worse results of re-balancing methods prove that they will hurt feature learning.

Therefore, in this paper, for exhaustively improving the recognition performance of long-tailed problems, we propose a unified Bilateral-Branch Network (BBN) model to take care of \emph{both representation learning and classifier learning} simultaneously. As shown in Figure~\ref{fig:structure}, our BBN model consists of two branches, termed as the ``conventional learning branch'' and the ``re-balancing branch''. In general, each branch of BBN separately performs its own duty for representation learning and classifier learning, respectively. As the name suggests, the conventional learning branch equipped with the typical uniform sampler w.r.t. the original data distribution is responsible for learning universal patterns for recognition. While, the re-balancing branch coupled with a reversed sampler is designed to model the tail data. After that, the predicted outputs of these bilateral branches are aggregated in the cumulative learning part by an adaptive trade-off parameter $\alpha$. $\alpha$ is automatically generated by the ``{Adaptor}'' according to the number of training epochs, which adjusts the whole BBN model to firstly learn the universal features from the original distribution and then pay attention to the tail data gradually. More importantly, $\alpha$ could further control the parameter updating of each branch, which, for example, avoids damaging the learned universal features when emphasizing the tail data at the later periods of training.

In experiments, empirical results on four benchmark long-tailed datasets show that our model obviously outperforms existing state-of-the-art methods. Moreover, extensive validation experiments and ablation studies can prove the aforementioned preliminary discovery and also validate the effectiveness of our tailored designs for long-tailed problems.

The main contributions of this paper are as follows:
\begin{itemize}[itemsep=-0.2em, leftmargin=1em]
	\item We explore the mechanism of the prominent class re-balancing methods for long-tailed problems, and further discover that these methods can significantly promote classifier learning and meanwhile will affect the representation learning w.r.t. the original data distribution.
	\item We propose a unified Bilateral-Branch Network (BBN) model to take care of both representation learning and classifier learning for exhaustively boosting long-tailed recognition. Also, a novel cumulative learning strategy is developed for adjusting the bilateral learnings and coupled with our BBN model's training.
	\item We evaluate our model on four benchmark long-tailed visual recognition datasets, and our proposed model consistently achieves superior performance over previous competing approaches.
\end{itemize}

\section{Related work}

\textbf{Class re-balancing strategies:} \emph{Re-sampling} methods as one of the most important class re-balancing strategies could be divided into two types: 1) Over-sampling by simply repeating data for minority classes~\cite{shen2016relay, buda2018systematic,  byrd2019effect} and 2) under-sampling by abandoning data for dominant classes~\cite{japkowicz2002class,buda2018systematic,he2009learning}. But sometimes, with re-sampling, duplicated tailed samples might lead to over-fitting upon minority classes~\cite{chawla2002smote,cb-focal}, while discarding precious data will certainly impair the generalization ability of deep networks.

\emph{Re-weighting} methods are another series of prominent class re-balancing strategies, which usually allocate large weights for training samples of tail classes in loss functions~\cite{huang2016learning, wang2017learning}. However, re-weighting is not capable of handling the large-scale, real-world scenarios of long-tailed data and tends to cause optimization difficulty~\cite{mikolov2013distributed}. Consequently, Cui~{\em et al.}~\cite{cb-focal} proposed to adopt the effective number of samples~\cite{cb-focal} instead of proportional frequency. Thereafter, Cao~{\em et al.}~\cite{ldam} explored the margins of the training examples and designed a label-distribution-aware loss to encourage larger margins for minority classes.

In addition, recently, some two-stage fine-tuning strategies~\cite{ldam, cui2018large, ouyang2016factors} were developed to modify re-balancing for effectively handling long-tailed problems. Specifically, they separated the training process into two single stages. In the first stage, they trained networks as usual on the original imbalanced data and only utilized re-balancing at the second stage to fine-tune the network with a small learning rate.

Beyond that, other methods of different learning paradigms were also proposed to deal with long-tailed problems, {\em e.g.}, metric learning~\cite{zhang2017range, huang2016learning}, meta-learning~\cite{Liu_2019_CVPR} and knowledge transfer learning~\cite{wang2017learning,Zhong_2019_CVPR}, which, however, are not within the scope of this paper.

\textbf{Mixup:}
Mixup~\cite{mixup} was a general data augmentation algorithm, {\em i.e.}, convexly combining random pairs of training images and their associated labels, to generate \emph{additional} samples when training deep networks. Also, manifold mixup~\cite{manifoldmixup} conducted mixup operations on random pairs of samples in the manifold feature space for augmentation. The mixed ratios in mixup were sampled from the $\beta$-distribution to increase the randomness of augmentation. Although mixup is clearly far from our unified end-to-end trainable model, in experiments, we still compared with a series of mixup algorithms to validate our effectiveness.

\section{How class re-balancing strategies work?}
\label{sec:how class re-balancing work}

In this section, we attempt to figure out the working mechanism of these class re-balancing methods. More concretely, we divide a deep classification model into two essential parts: 1) the feature extractor ({\em i.e.}, frontal base/backbone networks) and 2) the classifier ({\em i.e.}, last fully-connected layers). Accordingly, the learning process of a deep classification network could be separated into representation learning and classifier learning. Since class re-balancing strategies could boost the classification accuracy by altering the training data distribution closer to test and paying more attention to the tail classes, we propose a conjecture that the way these strategies work is to promote classifier learning significantly but might damage the universal representative ability of the learned deep features due to distorting original distributions.

In order to justify our conjecture, we design a two-stage experimental fashion to separately learn representations and classifiers of deep models. Concretely, in the first stage, we train a classification network with plain training ({\em i.e.}, cross-entropy) or re-balancing methods ({\em i.e.}, re-weighting/re-sampling) as learning manners. Then, we obtain different kinds of feature extractors corresponding to these learning manners. When it comes to the second stage, we fix the parameters of the feature extractors learned in the former stage, and retrain classifiers \textit{from scratch} with the aforementioned learning manners again. In principle, we design these experiments to fairly compare the quality of representations and classifiers learned by different manners by following the control variates method.

The CIFAR~\cite{cifar} datasets are a collection of images that are commonly used to assess computer vision approaches. Previous work~\cite{cb-focal,ldam} created long-tailed versions of CIFAR datasets with different imbalance ratios, {\em i.e.}, the number of the most frequent class divided by the least frequent class, to evaluate the performance. In this section, following~\cite{ldam}, we also use long-tailed CIFAR-10/CIFAR-100 as the test beds.

As shown in Figure~\ref{fig:representation and classifier learning}, we conduct several contrast experiments to validate our conjecture on CIFAR-100-IR50 (long-tailed CIFAR-100 with imbalance ratio $50$). As aforementioned, we separate the whole network into two parts: the feature extractor and classifier. Then, we apply three manners for the feature learning and the classifier learning respectively according to our two-stage training fashion. Thus, we can obtain nine groups of results based on different permutations: (1) Cross-Entropy (CE): We train the networks as usual on the original imbalanced data with the conventional cross-entropy loss. (2) Re-Sampling (RS): We first sample a class uniformly and then collect an example from that class by sampling with replacement. By repeating this process, a balanced mini-batch data is obtained. (3) Re-Weighting (RW): We re-weight all the samples by the inverse of the sample size of their classes. The error rate is evaluated on the validation set. As shown in Figure~\ref{fig:representation and classifier learning}, we have the observations from two perspectives:
\begin{itemize}[itemsep=-0.2em, leftmargin=1em]
        \item \textbf{Classifiers:} When we apply the same representation learning manner (comparing error rates of three blocks in the vertical direction), it can be reasonably found that RW/RS always achieve lower classification error rates than CE, which owes to their re-balancing operations adjusting the classifier weights' updating to match test distributions.
	\item \textbf{Representations:} When applying the same classifier learning manner (comparing error rates of three blocks in the horizontal direction), it is a bit of surprise to see that error rates of CE blocks are consistently lower than error rates of RW/RS blocks. The findings indicate that training with CE achieves better classification results since it obtains better features. The worse results of RW/RS reveal that they lead to inferior discriminative ability of the learned deep features.
\end{itemize}

Furthermore, as shown in Figure~\ref{fig:representation and classifier learning} (left), by employing CE on the representation learning and employing RS on the classifier learning, we can achieve the lowest error rate on the validation set of CIFAR-100-IR50. Additionally, to evaluate the generalization ability for representations produced by three manners, we utilize pre-trained models trained on CIFAR-\emph{100}-IR50 as the feature extractor to obtain the representations of CIFAR-\emph{10}-IR50, and then perform the classifier learning experiments as the same as aforementioned. As shown in Figure~\ref{fig:representation and classifier learning} (right), on CIFAR-10-IR50, it can have the identical observations, even in the situation that the feature extractor is trained on another long-tailed dataset.

\section{Methodology}

\subsection{Overall framework}

As shown in Figure~\ref{fig:structure}, our BBN consists of three main components. Concretely, we design two branches for representation learning and classifier learning, termed ``\textit{conventional learning branch}'' and ``\textit{re-balancing branch}'', respectively. Both branches use the same residual network structure~\cite{he2016deep} and share all the weights except for the last residual block. Let $\mathbf{x}_{\cdot}$ denote a training sample and ${y}_{\cdot}\in\{1,2,\ldots,C\}$ is its corresponding label, where $C$ is the number of classes. For the bilateral branches, we apply uniform and reversed samplers to each of them separately and obtain two samples $(\mathbf{x}_{c}, {y}_{c})$ and $(\mathbf{x}_{r}, {y}_{r})$ as the input data, where $(\mathbf{x}_{c}, {y}_{c})$ is for the conventional learning branch and $(\mathbf{x}_{r}, {y}_{r})$ is for the re-balancing branch. Then, two samples are fed into their own corresponding branch to acquire the feature vectors $\bm{f}_c \in \mathbb{R}^{D}$ and $\bm{f}_r \in \mathbb{R}^{D}$ by global average pooling.

Furthermore, we also design a specific cumulative learning strategy for shifting the learning ``attention'' between two branches in the training phase. In concretely, by controlling the weights for $\bm{f}_c$ and $\bm{f}_r$ with an adaptive trade-off parameter $\alpha$, the weighted feature vectors $\alpha \bm{f}_c$ and $(1-\alpha) \bm{f}_r$ will be sent into the classifiers $\bm{W}_c \in \mathbb{R}^{D\times C}$ and $\bm{W}_r \in \mathbb{R}^{D\times C}$ respectively and the outputs will be integrated together by element-wise addition. The output logits are formulated as
\begin{equation}
\label{output}
\mathbf{z} = \alpha \bm{W}^{\top}_c \bm{f}_c + (1-\alpha) \bm{W}^{\top}_r  	\bm{f}_r,
\end{equation}
where $\mathbf{z}  \in \mathbb{R}^{C}$ is the predicted output, {\em i.e.}, $[z_1,z_2,\ldots, z_C ]^{\top}$. For each class $i \in \left\lbrace 1, 2, \ldots, C\right\rbrace$, the softmax function calculates the probability of the class by
\begin{equation}
\label{softmax}
\hat{p}_i = \dfrac{e^{z_i}}{\sum_{j=1}^{C} e^{z_j}}.
\end{equation}

Then, we denote $E(\cdot,\cdot)$ as the cross-entropy loss function and the output probability distribution as $\hat{\bm{p}}=[\hat{p}_1,\hat{p}_2,..., \hat{p}_C ]^{\top}$. Thus, the weighted cross-entropy classification loss of our BBN model is illustrated as
\begin{equation}
\label{loss}
\mathcal{L} = \alpha  E({\hat{\bm{p}}, {y}_{c}}) + (1-\alpha)  E({\hat{\bm{p}}, {y}_{r}}),
\end{equation}
and the whole network is end-to-end trainable.

\subsection{Proposed bilateral-branch structure}

In this section, we elaborate the details of our unified bilateral-branch structure shown in Figure~\ref{fig:structure}. As aforementioned, the proposed {conventional learning branch} and {re-balancing branch} do perform their own duty ({\em i.e.}, representation learning and classifier learning, respectively). There are two unique designs for these branches.

{\textbf{Data samplers. }}
The input data for the conventional learning branch comes from a uniform sampler, where each sample in the training dataset is sampled only once with equal probability in a training epoch. The uniform sampler retains the characteristics of original distributions, and therefore benefits the representation learning. While, the re-balancing branch aims to alleviate the extreme imbalance and particularly improve the classification accuracy on tail classes~\cite{van2017devil}, whose input data comes from a reversed sampler. For the reversed sampler, the sampling possibility of each class is proportional to the reciprocal of its sample size, {\em i.e.}, the more samples in a class, the smaller sampling possibility that class has. In formulations, let denote that the number of samples for class $i$ is $N_i$ and the maximum sample number of all the classes is $N_{max}$. There are three sub-procedures to construct the reversed sampler: 1) Calculate the sampling possibility $P_i$ for class $i$ according to the number of samples as
\begin{equation}
\label{weight}
\begin{aligned}
P_i = \frac{w_i}{\sum_{j=1}^{C} w_j}\,,
\end{aligned}
\end{equation}
where $w_i=\frac{N_{max}}{N_i}$; 2) Randomly sample a class according to $P_i$; 3) Uniformly pick up a sample from class $i$ with replacement. By repeating this reversed sampling process, training data of a mini-batch is obtained.

{\textbf{Weights sharing. }}
In BBN, both branches economically share the same residual network structure, as illustrated in Figure~\ref{fig:structure}. We use ResNets~\cite{he2016deep} as our backbone network, {\em e.g.}, ResNet-32 and ResNet-50. In details, two branch networks, except for the last residual block, share the same weights. There are two benefits for sharing weights: On the one hand, the well-learned representation by the conventional learning branch can benefit the learning of the re-balancing branch. On the other hand, sharing weights will largely reduce computational complexity in the inference phase.

\subsection{Proposed cumulative learning strategy}\label{sec:cumulative}

Cumulative learning strategy is proposed to shift the learning focus between the bilateral branches by controlling both the weights for features produced by two branches and the classification loss $\mathcal{L}$. It is designed to first learn the universal patterns and then pay attention to the tail data gradually. In the training phase, the feature $\bm{f}_c$ of the conventional learning branch will be multiplied by $\alpha$ and the feature $\bm{f}_r$ of the re-balancing branch will be multiplied by $1-\alpha$, where $\alpha$ is automatically generated according to the training epoch. Concretely, the number of total training epochs is denoted as $T_{max}$ and the current epoch is $T$. $\alpha$ is calculated by
\begin{equation}
\label{alpha}
\alpha = 1 - \left(\frac{T}{T_{max}}\right)^{2},
\end{equation}
which $\alpha$ will gradually decrease as the training epochs increasing.

\begin{table*}[t]
\footnotesize
\caption{Top-1 error rates of ResNet-32 on long-tailed CIFAR-10 and CIFAR-100. (Best results are marked in bold.) }
\vspace{2mm}
\begin{center}
	\begin{tabular}{|c|c|c|c|c|c|c|}
		\hline
		Dataset                      & \multicolumn{3}{c|}{Long-tailed CIFAR-10}                             & \multicolumn{3}{c|}{Long-tailed CIFAR-100}                             \\ \hline
		Imbalance ratio              & \multicolumn{1}{c|}{100} & \multicolumn{1}{c|}{50} & 10             & \multicolumn{1}{c|}{100} & \multicolumn{1}{c|}{50} & 10      \\
		\hline
		\hline
		CE                           & 29.64                    & 25.19                   & 13.61          & 61.68                    & 56.15                   & 44.29          \\
		Focal~\cite{focalloss}                        & 29.62                    & 23.28                   & 13.34          & 61.59                    & 55.68                   & 44.22          \\
		Mixup~\cite{mixup}                     & 26.94                    & 22.18                   & 12.90          & 60.46                    & 55.01                   & 41.98          \\
		Manifold Mixup~\cite{manifoldmixup}              & 27.04                    & 22.05                   & 12.97          & 61.75                    & 56.91                   & 43.45          \\
		Manifold Mixup (two samplers) & 26.90                    & 20.79                   & 13.17          & 63.19                    & 57.95                   & 43.54          \\ \hline
		CE-DRW~\cite{ldam}                    & 23.66                    & 20.03                   & 12.44           & 58.49                    & 54.71                   & 41.88         \\
		CE-DRS~\cite{ldam}                     & 24.39                    & 20.19                   & 12.62          & 58.39                    & 54.52                  & 41.89          \\ \hline
		CB-Focal~\cite{cb-focal}                     & 25.43                    & 20.73                   & 12.90           & 60.40                     & 54.83                   & 42.01          \\
		LDAM-DRW~\cite{ldam}                     & 22.97                    & 18.97                   & 11.84          & 57.96                    & 53.38                   & 41.29          \\ \hline
		\hline
		Our BBN                        & \textbf{20.18}           & \textbf{17.82}          & \textbf{11.68} & \textbf{57.44}           & \textbf{52.98}          & \textbf{40.88} \\ \hline
	\end{tabular}
\end{center}
\label{tab:cifar_results}
\end{table*}

In intuition, we design the adapting strategy for $\alpha$ based on the motivation that discriminative feature representations are the foundation for learning robust classifiers. Although representation learning and classifier learning deserve equal attentions, the learning focus of our BBN should gradually change from feature representations to classifiers, which can exhaustively improve long-tailed recognition accuracy. With $\alpha$ decreasing, the main emphasis of BBN turns from the conventional learning branch to the re-balancing branch. Different from two-stage fine-tuning strategies~\cite{ldam, cui2018large, ouyang2016factors}, our $\alpha$ ensures that both branches for different goals can be constantly updated in the whole training process, which could avoid the affects on one goal when it performs training for the other goal.

In experiments, we also provide the qualitative results of this intuition by comparing different kinds of adaptors, cf. Section~\ref{sec:alpha}.

\subsection{Inference phase}
During inference, the test samples are fed into both branches and two features $\bm{f}'_c$ and $\bm{f}'_r$ are obtained. Because both branches are equally important, we simply fix $\alpha$ to $0.5$ in the test phase. Then, the equally weighted features are fed to their corresponding classifiers ({\em i.e.}, $\bm{W}_c$ and $\bm{W}_r$) to obtain two prediction logits. Finally, both logits are aggregated by element-wise addition to return the classification results

\section{Experiments}

\subsection{Datasets and empirical settings}\label{sec:datasets}

{\noindent \textbf{Long-tailed CIFAR-10 and CIFAR-100. }}
Both CIFAR-10 and CIFAR-100 contain 60,000 images, 50,000 for training and 10,000 for validation with category number of 10 and 100, respectively. For fair comparisons, we use the long-tailed versions of CIFAR datasets as the same as those used in~\cite{ldam} with controllable degrees of data imbalance. We use an imbalance factor $\beta$ to describe the severity of the long tail problem with the number of training samples for the most frequent class and the least frequent class, {\em e.g.}, $\beta=\frac{N_{max}}{N_{min}}$. Imbalance factors we use in experiments are $10$, $50$ and $100$.

{\noindent \textbf{iNaturalist 2017 and iNaturalist 2018. }}
The iNaturalist species classification datasets are large-scale real-world datasets that suffer from extremely imbalanced label distributions. The 2017 version of iNaturalist contains 579,184 images with 5,089 categories and the 2018 version is composed of 437,513 images from 8,142 categories. Note that, besides the extreme imbalance, the iNaturalist datasets also face the fine-grained problem~\cite{wei2019deep,Zhao2017,scda,pcm}. In this paper, the official splits of training and validation images are utilized for fair comparisons.

\subsection{Implementation details}

{\noindent \textbf{Implementation details on CIFAR. }}
For long-tailed CIFAR-10 and CIFAR-100 datasets, we follow the data augmentation strategies proposed in~\cite{he2016deep}: randomly crop a $32\times32$ patch from the original image or its horizontal flip with $4$ pixels padded on each side. We train the ResNet-32~\cite{he2016deep} as our backbone network for all experiments by standard mini-batch stochastic gradient descent (SGD) with momentum of $0.9$, weight decay of $2\times{10}^{-4}$. We train all the models on a single NVIDIA 1080Ti GPU for $200$ epochs with batch size of $128$. The initial learning rate is set to $0.1$ and the first five epochs is trained with the linear warm-up learning rate schedule~\cite{goyal2017accurate}. The learning rate is decayed at the $120^{\rm th}$ and $160^{\rm th}$ epoch by $0.01$ for our BBN.

{\noindent \textbf{Implementation details on iNaturalist. }}
For fair comparisons, we utilize ResNet-50~\cite{he2016deep} as our backbone network in all experiments on iNaturalist 2017 and iNaturalist 2018. We follow the same training strategy in~\cite{goyal2017accurate} with batch size of $128$ on four GPUs of NVIDIA 1080Ti. We firstly resize the image by setting the shorter side to $256$ pixels and then take a $224 \times 224$ crop from it or its horizontal flip. During training, we decay the learning rate at the $60^{\rm th}$ and $80^{\rm th}$ epoch by $0.1$ for our BBN, respectively.

\subsection{Comparison methods}
In experiments, we compare our BBN model with three groups of methods:

\begin{itemize}[itemsep=-0.2em, leftmargin=1em]
	\item {\noindent \textbf{Baseline methods.}} We employ plaining training with cross-entropy loss and focal loss~\cite{focalloss} as our baselines. Note that, we also conduct experiments with a series of mixup algorithms~\cite{mixup,manifoldmixup} for comparisons.
	\item {\noindent \textbf{Two-stage fine-tuning strategies.}}
	To prove the effectiveness of our cumulative learning strategy, we also compare with the two-stage fine-tuning strategies proposed in previous state-of-the-art~\cite{ldam}. We train networks with cross-entropy (CE) on imbalanced data in the first stage, and then conduct class re-balancing training in the second stage. ``CE-DRW'' and ``CE-DRS'' refer to the two-stage baselines using re-weighting and re-sampling at the second stage.
	\item {\noindent \textbf{State-of-the-art methods.}} For state-of-the-art methods, we compare with the recently proposed LDAM~\cite{ldam} and CB-Focal~\cite{cb-focal} which achieve good classification accuracy on these four aforementioned long-tailed datasets.
\end{itemize}

\begin{table}[t]
	\footnotesize
	\renewcommand\arraystretch{1.1}
	\caption{Top-1 error rates of ResNet-50 on large-scale long-tailed datasets iNaturalist 2018 and iNaturalist 2017. Our method outperforms the previous state-of-the-arts by a large margin, especially with {$2\times$} scheduler. ``*'' indicates original results in that paper.}
	\vspace{2mm}
	\centering
	\begin{tabular}{|c|c|c|}
		\hline
		Dataset        & iNaturalist 2018 & iNaturalist 2017           \\ \hline\hline
		CE              & 42.84  & 45.38            \\
		\hline
		CE-DRW~\cite{ldam}          & 36.27  &  40.48          \\
		CE-DRS~\cite{ldam}        & 36.44  &   40.12          \\
		\hline
		CB-Focal~\cite{cb-focal}        & 38.88  & 41.92            \\
		LDAM-DRW*~\cite{ldam}       & 32.00  & --            \\
		LDAM-DRW~\cite{ldam}        & 35.42  & 39.49            \\
		LDAM-DRW~\cite{ldam} ($2\times$)    & 33.88  & 38.19 \\
		\hline
		\hline
		Our BBN             & {33.71}  & {36.61}             \\
		Our BBN ($2\times$)         & \textbf{30.38}  &  \textbf{34.25}     \\ \hline
	\end{tabular}
	\label{tab:inat_results}
\end{table}

\subsection{Main results}

\subsubsection{Experimental results on long-tailed CIFAR}

We conduct extensive experiments on long-tailed CIFAR datasets with three different imbalanced ratios: $10$, $50$ and $100$. Table~\ref{tab:cifar_results} reports the error rates of various methods. We demonstrate that our BBN consistently achieves the best results across all the datasets, when comparing other comparison methods, including the two-stage fine-tuning strategies ({\em i.e.}, CE-DRW/CE-DRS), the series of mixup algorithms ({\em i.e.}, mixup, manifold mixup and manifold mixup with two samplers as the same as ours), and also previous state-of-the-arts ({\em i.e.}, CB-Focal~\cite{cb-focal} and LDAM-DRW~\cite{ldam}).

Especially for long-tailed CIFAR-10 with imbalanced ratio $100$ (an extreme imbalance case), we get 20.18\% error rate which is 2.79\% lower than that of LDAM-DRW~\cite{ldam}. Additionally, it can be found from that table, the two-stage fine-tuning strategies ({\em i.e.}, CE-DRW/CE-DRS) are effective, since they could obtain comparable or even better results comparing with state-of-the-art methods.

\subsubsection{Experimental results on iNaturalist}

Table~\ref{tab:inat_results} shows the results on two large-scale long-tailed datasets, {\em i.e.}, iNaturalist 2018 and iNaturalist 2017. As shown in that table, the two-stage fine-tuning strategies ({\em i.e.}, CE-DRW/CE-DRS) also perform well, which have consistent observations with those on long-tailed CIFAR. Compared with other methods, on iNaturalist, our BBN still outperform competing approaches and baselines. Besides, since iNaturalist is large-scale, we also conduct network training with the $2\times$ scheduler. Meanwhile, for fair comparisons, we further evaluate the previous state-of-the-art LDAM-DRW~\cite{ldam} with the $2\times$ training scheduler. It is obviously to see that, with $2\times$ scheduler, our BBN achieves significantly better results than BBN without $2\times$ scheduler. Additionally, compared with LDAM-DRW ($2\times$), we achieve $+3.50\%$ and $+3.94\%$ improvements on iNaturalist 2018 and iNaturalist 2017, respectively. In addition, even though we do not use the $2\times$ scheduler, our BBN can still get the best results. For a detail, we conducted the experiments based on LDAM~\cite{ldam} with the source codes provided by the authors, but failed to reproduce the results reported in that paper.

\subsection{Ablation studies}
\subsubsection{Different samplers for the re-balancing branch}

For better understanding our proposed BBN model, we conduct experiments on different samplers utilized in the re-balancing branch. We present the error rates of models trained with different samplers in Table~\ref{tab:sampler}. For clarity, the uniform sampler maintains the original long-tailed distribution. The balanced sampler assigns the same sampling possibility to all classes, and construct a mini-batch training data obeying a balanced label distribution. As shown in that table, the reversed sampler (our proposal) achieves considerably better performance than the uniform and balanced samplers, which indicates that the re-balancing branch of BBN should pay more attention to the tail classes by enjoying the reversed sampler.

\begin{table}[t]
	\footnotesize
	\caption{Ablation studies for different samplers for the re-balancing branch of BBN on long-tailed CIFAR-10-IR50.}
	\vspace{2mm}
	\centering
	\begin{tabular}{|c|c|}
		\hline
		Sampler  &  Error rate \\ \hline\hline
		Uniform sampler &  21.31               \\ \hline
		Balanced sampler &  21.06                 \\ \hline
		\hline
		Reversed sampler (Ours) &  \textbf{17.82}      \\ \hline
	\end{tabular}
	\label{tab:sampler}
\end{table}

\subsubsection{Different cumulative learning strategies}
\label{sec:alpha}

To facilitate the understanding of our proposed cumulative learning strategy, we explore several different strategies to generate the adaptive trade-off parameter $\alpha$ on CIFAR-10-IR50. Specifically, we test with both progress-relevant/irrelevant strategies, cf. Table~\ref{tab:adaptor}. For clarity, progress-relevant strategies adjust $\alpha$ with the number of training epochs, {\em e.g.}, linear decay, cosine decay, \etc. Progress-irrelevant strategies include the equal weight or generate from a discrete distribution ({\em e.g.}, the $\beta$-distribution).

\begin{table}[t]
\footnotesize
\renewcommand\arraystretch{1.5}
\caption{Ablation studies of different adaptor strategies of BBN on long-tailed CIFAR-10-IR50.}
\vspace{2mm}
\centering
\begin{tabular}{|c|c|c|}
	\hline
	Adaptor  & $\alpha$   & Error rate \\ \hline\hline
	Equal weight & $0.5$     &    21.56              \\ \hline
	$\beta$-distribution & $Beta(0.2, 0.2)$     & 21.75                \\ \hline
	Parabolic increment & $\left(\frac{T}{T_{max}}\right)^{2}$  & 22.70      \\ \hline
	Linear decay & $1 - \frac{T}{T_{max}}$     &       18.55         \\ \hline
	Cosine decay & $\cos({\frac{T}{T_{max}}}\cdot\frac{\pi}{2})$     &    18.04             \\ \hline
	\hline
	Parabolic decay (Ours) & $1 - \left(\frac{T}{T_{max}}\right)^{2}$  & \textbf{17.82} \\ \hline
\end{tabular}
\label{tab:adaptor}
\end{table}

\iffalse
\begin{figure}[t]
	\centering
	\includegraphics[width=0.65\linewidth]{fig/alpha}

	\caption{Different adaptor strategies for generating $\alpha$. The horizontal axis is current epoch ratio $\frac{T}{T_{max}}$ and the vertical axis indicates the value of $\alpha$.}
	\label{fig:alpha}
\end{figure}
\fi

As shown in Table~\ref{tab:adaptor}, the decay strategies ({\em i.e.}, linear decay, cosine decay and our parabolic decay) for generating $\alpha$ can yield better results than the other strategies ({\em i.e.}, equal weight, $\beta$-distribution and parabolic increment). These observations prove our motivation that the conventional learning branch should be learned firstly and then the re-balancing branch. Among these strategies, the best way for generating $\alpha$ is the proposed parabolic decay approach. In addition, the parabolic increment, where re-balancing are attended before conventional learning, performs the worst, which validates our proposal from another perspective. More detailed discussions can be found in the supplementary materials.

\subsection{Validation experiments of our proposals}

\subsubsection{Evaluations of feature quality}

It is proven in Section~\ref{sec:how class re-balancing work} that learning with vanilla CE on original data distribution can obtain good feature representations.
In this subsection, we further explore the representation quality of our proposed BBN by following the empirical settings in Section~\ref{sec:how class re-balancing work}. Concretely, given a BBN model trained on CIFAR-100-IR50, firstly, we fix the parameters of representation learning of two branches. Then, we separately retrain the corresponding classifiers from scratch of two branches also on CIFAR-100-IR50. Finally, classification error rates are tested on these two branches independently.

As shown in Table~\ref{tab:representation quality of BBN}, the feature representations obtained by the conventional learning branch of BBN (``BBN-CB'') achieves comparable performance with CE, which indicates that our proposed BBN greatly preserves the representation capacity learned from the original long-tailed dataset.
Note that, the re-balancing branch of BBN (``BBN-RB'') also gets better performance than RW/RS and it possibly owes to the parameters sharing design of our model.

\begin{table}[t]
	\footnotesize
	\caption{Feature quality evaluation for different learning manners.

	}
	\vspace{2mm}
	\centering
	\begin{tabular}{|cc|}
		\hline
		\multicolumn{1}{|c|}{Representation learning manner} & Error rate \\ \hline\hline
		\multicolumn{1}{|c|}{CE}               & \textbf{58.62}      \\
		\multicolumn{1}{|c|}{RW}               & 63.17      \\
		\multicolumn{1}{|c|}{RS}               & 63.71      \\ \hline\hline
		\multicolumn{1}{|c|}{BBN-CB}           & 58.89      \\
		\multicolumn{1}{|c|}{BBN-RB}           & 61.09      \\ \hline
	\end{tabular}
	\label{tab:representation quality of BBN}
\end{table}

\subsubsection{Visualization of classifier weights}

Let denote $\bm{W} \in \mathbb{R}^{D\times C}$ as a set of classifiers $\{\bm{w}_1,\bm{w}_2,..., \bm{w}_C\}$ for all the $C$ classes, where $\bm{w}_i \in \mathbb{R}^{D}$ indicates the weight vector for class $i$. Previous work~\cite{guo2017one} has shown that the value of $\ell_2$-norm ${\left\lbrace\|\bm{w}_i\|_2\right\rbrace }_{i=1}^{C}$ for different classes can demonstrate the preference of a classifier, {\em i.e.}, the classifier $\bm{w}_i$ with the largest $\ell_2$-norm tends to judge one example as belonging to its class $i$. Following~\cite{guo2017one}, we visualize the $\ell_2$-norm of these classifiers.

\begin{figure}[t]
	\centering
	\includegraphics[width=0.695\linewidth]{fig/weight_vis_new_std}

	\caption{$\ell_2$-norm of classifier weights for different learning manners. Specifically, ``BBN-ALL'' indicates the $\ell_2$-norm of the combination of $\bm{W}_c$ and $\bm{W}_r$ in our model. $\sigma$ in the legend is the standard deviation of $\ell_2$-norm for ten classes.}

	\label{fig:weight vis}
\end{figure}

As shown in Figure~\ref{fig:weight vis}, we visualize the $\ell_2$-norm of ten classes trained on CIFAR-10-IR50. For our BBN, we visualize the classifier weights $\bm{W}_{c}$ of the conventional learning branch (``BBN-CB'') and the classifier weights $\bm{W}_{r}$ of the re-balancing branch (``BBN-RB''), as well as their combined classifier weights (``BBN-ALL''). Additionally, the visualization results on classifiers trained with these learning manners in Section~\ref{sec:how class re-balancing work}, {\em i.e.}, CE, RW and RS, are also provided.

Obviously, the $\ell_2$-norm of ten classes' classifiers for our proposed model ({\em i.e.}, ``BBN-ALL'') are basically equal, and their standard deviation $\sigma=0.148$ is the smallest one. For the classifiers trained by other learning manners, the distribution of the $\ell_2$-norm of CE is consistent with the long-tailed distribution. The $\ell_2$-norm distribution of RW/RS looks a bit flat, but their standard deviations are larger than ours. It gives an explanation why our BBN can outperform these methods. Additionally, by separately analyzing our model, its conventional learning branch (``BBN-CB'') has a similar $\ell_2$-norm distribution with CE's, which justifies its duty is focusing on universal feature learning. The $\ell_2$-norm distribution of the re-balancing branch (``BBN-RB'') has a reversed distribution w.r.t. original long-tailed distributions, which reveals it is able to model the tail.

\section{Conclusions}

In this paper, for studying long-tailed problems, we explored how class re-balancing strategies influenced representation learning and classifier learning of deep networks, and revealed that they can promote classifier learning significantly but also damage representation learning to some extent. Motivated by this, we proposed a Bilateral-Branch Network (BBN) with a specific cumulative learning strategy to take care of both representation learning and classifier learning for exhaustively improving the recognition performance of long-tailed tasks. By conducting extensive experiments, we proved that our BBN could achieve the best results on long-tailed benchmarks, including the large-scale iNaturalist. In the future, we attempt to tackle the long-tailed detection problems with our BBN model.

\clearpage

\onecolumn

redpajama_set_name: RedPajamaArXiv


Every year we used to labor to find an employee who has just the right blend of computer skills coupled with a background in photography. Three years ago, we hit the jackpot with our technology director, Wendy Perry. She is an extremely talented photographer and longtime camp lady who is excited about the opportunity to maintain our website and assist you in viewing what is going on here with your campers!
So why do we run this website? Because we want you to be able to be caught up in the story of Camp Timberlake! By doing this we are giving you a one way mirror into what all we do here while camp is in session. As you will see we have a lot of fun! Chances are that you will see your boy on this website in the coming days, but if not, please be patient with us. It takes a while to get everyone on camera. We are going to do quite a few "cabin shots" to try to help with this. But I can promise that you will be able to know what all is going on here and in some cases before the campers even do. If you are having trouble navigating it or logging on please don't hesitate to give us a call.
Thanks again for sending your boys to Camp Timberlake and we want you to know how proud we are to have you in our camp family!


This is not a necessary upgrade for my car, which is a 2000 Mitsubishi Galant ES, with tein s tech lowering springs and kyb struts installed, front 1.7 & rear 1.3 inches lowered. This product fits well with the setup, of course the length has been adjusted to shorter than stock, but the mount is good. I don't usually get the work done by myself, so I just took the car the a local workshop, seems easy for them. Reason why it's not 5 stars is just because I don't think I have test drove the car hard enough to actually see a different, but in general, I'm very satisfied, the silver color does stands out.


Our desires must be focused on Him: Christ our Lord. We must feel the anticipation and the excitement to be with and in Him, so that we not only hear His Word; but we do His Word, the Word of God.
Our desires must be focused on Him: Christ our Lord. We must feel the anticipation and the excitement to be with and in Him, so that we not only hear His Word; but we do His Word, the Word of God. So if we become stuck in our anxious thoughts, confusion, or the stressed out urgency of life, we stop and get our focus right! Pray that the Holy Sprit intercedes in you and removes the disruptions. Allow yourself to receive His comfort and grace. Remember it is not anything we do or effort on our part, we are only to receive what Christ our Lord gives us. And to receive, we need to make sure stuff that gets in the way, are out of the way.
Augustine, a "Catholic Saint," was one of the foremost philosophers and theologians of early Christianity. While serving (396-430) as bishop of Hippo Regius, the leading figure in the church, he was perhaps the greatest Christian thinker of all time and the one person with the biggest influence in theology for Catholics and Protestants, even more than the reformers. He also was the main person who influenced the "western world view," our cultural and identity both in Europe and the U.S.!
Augustine fought vigorously against his bad habits and sin nature, so he can grow deeper in our Lord. Until he learned a lesson from a small child, which he felt was the Word of God. Let Augustine's influence keep influencing you!
There are times when I journal, that is keep a diary of what is going on in my life. I sometimes look back on them and gain new insight on what I was going through, that I did not have back then. I have become more adept to interpret God's leading and plan for me the more I experience life and receive what He has done. By looking back on what we have been through and see the hand of God there, it will give great comfort and encouragement for what lies ahead or what we are going through now.
As I gain new insight into my personality and the Lord's working into my issues and problems, I realize how shadows they are compared to my Lord's holiness and greatness.
By concentrating on Christ and what He has done will lift us up better and more complete than anything we could ever do. Thus the thrust of journaling has taught me is to look beyond myself and keep focused on Christ. The other end of journaling is a problem that people, including Christians, in their zeal keep faithful to their journals, is that they become self absorbed and only see themselves and their problems and not the Lord. So be careful if you do this exercise of journaling, keep focused on why you are doing it, that is to grow closer to the Lord and not to yourself.
We are called to keep focused on Him and not ourselves, so we are looking at God's Word as a mirror to ourselves, to our soul, as not to see us, but to see God working in us. When we only see ourselves, we see sin, brokenness, failure, self-seeking inclinations, and wrong attitudes.
We must see God's interests and not our own, then the journey of maturing in the faith will become more real as our problems become less, He becomes more.
The same thing can happen when we read the Word. We become so consumed with our interests, we do not see the calling and response we are to give. Thus we grow bitter, thinking that this devotion stuff is not for me, so we turn it off. We replace it with so much activity that God is pushed out of our lives, except on Sunday morning. But even then we are rushed and stressed and do not feel the worship or hear the lesson. We only hear ourselves, our problems of getting the kids ready, or the stress at work or at school. The results of a mature life will respond from the impact of our devotional life, by applying what Christ has done.
We need to respond to the text with a surrendered will and a mind cleared of our anxious thoughts. When we are focused on our fears, hopes, dreams, needs, or emotions, we have no room to learn what God has for us. We will not be able to think deeply enough into the text so there is a transformation of our nature and will, as philosophers call our existential core. There can be no serious behavior or personality change unless the core of who we are changes. And Christ is the only one who does that right!
This transformation in Romans 12 cannot happen when we are in the way. God does as He pleases, but He usually does not override our will. He waits for us to be surrendered and poured out to Him. So do not take the chance and allow your stubbornness to get in the way of God working in you!