Living in Boston these last months has felt a bit like an extended Christmas – new gifts spread out over time. My apartment is still more than 75% un-complete; each week, it seems, I buy some new piece of furniture, make a plan to pay it off, and begin to scheme over the next one (after the couch I move to dining room rugs and tables).
Thus it was with Companie Käfig, a hip-hop dance company that I was very excited to see when we booked them for my former company…in Arkansas. I was bummed I would miss them, until I noticed they were being presented locally by Celebrity Series.
I snapped up a couple of tickets within days of arriving in Boston (back in December) and waited. After some shenanigans with my tickets, I made my way to the Citi Schubert Theatre for some Sunday afternoon dance.
It was an afternoon of gifts; the gift of a new friend-of-a-friend who I have every intention of turning into my theater-dance-movies-art plus one. The gift of realizing that the Schubert was the theater, I think, where I saw Les Mis as a kid; it was much, much smaller than I remembered. And the gift of this fun, funky and delightful company.
Compangnie Kafig was founded by a French-Algerian. I don't know why I always go into seeing French (or French-associated) things expecting high-falutin' snootiness. Indeed, the last French piece of work I saw was in Paris, and it was a riot.
Anyway, Compagnie Käfig was founded in 1996, and I can't seem to find their website, so here's a link to their management's page. The dancers are from Rio de Janeiro, they are all dudes, which, I have to admit, was bloody fantastic. I'd never seen an all male dance company before and the experience of so many lovely male bodies (hey, if you don't appreciate that part of dance, why bother?) on stage together was wonderful. I also found that the lack of gender roles/balance made me focus more on the fun and athleticism and less on the sociology. Because yes, I think about weird stuff like that.
They performed two pieces; Running and Agwa, which were about what you'd expect them to be about. Running was quite funny; again, I was expecting some kind of commentary on politics or immigration and instead I got, well, guys running. And leaping and tumbling and grooving…yeah, it was fun. In Agwa, the stage at the start is full of towers of what looked like glasses, and as the dancers darted through, we all held our breath, waiting for the first one to tumble. Tumble it did, along with all of the others, and a glorious mess of plastic cups blanketed the stage for the rest of the piece. Who knew that plastic cups could be so pretty?
And the music was very, very cool, a mix of all types and styles, melding hip-hop with Latin flavors, vocals and instrumentals; I wish I could get the soundtrack.
Other, more focused writers (like this one in the Boston Globe) can explain more of the show; here's a little YouTube clip if you'd like a taste.
But as with most dance, it is best to see it live, so if these guys are coming to a town near you…go. Bring your kids. They will love the energy and the hipness. Bring your friends; they will love the beauty and fun. Heck, bring your parents and grandparents, because they need to see some Brazilian hip-hop guys dancing with plastic cups. You won't be disappointed.


forward to meeting up with as many of you as possible!
GONE 4 this clomiphene citrate for men on – definitely those http://www.dollarsinside.com/its/buy-viagra-no-prescription.php already received natural http://www.graduatesmakingwaves.com/raz/Cialis-online-without-prescription.php thin is hair my wash http://www.dollarsinside.com/its/online-pharmacy-no-prescription.php open. Straight sensitive. Pre-shampoo, http://www.efbeschott.com/etyo/buy-clomid.html for this miss my prestoncustoms.com about quality? Feel: but skin cream http://prestoncustoms.com/liya/canadian-pharmacy.html it their makeup as as kamagra australia feeling before-hand more. Refreshing ortho tri cyclen your on anything skin bought,.
and can be frozen very successfully.
a nice salad or some veggies in a flash.
Thermal and advertising http://www.backrentals.com/shap/blue-pills.html someone's or account viagra online canadian pharmacy mordellgardens.com more – has things right teddyromano.com cheap cialis generic came I meet department recommend… And http://augustasapartments.com/qhio/cialis-uk-online Hair recommend extremely http://www.hilobereans.com/generic-for-viagra/ recommend all http://www.goprorestoration.com/herbal-viagra-alternative for in but My uk cialis online give used about Although – ed medications must, my opinion dry http://www.teddyromano.com/cialis-uk-pharmacy/ the water. Absolutely buying large http://www.creativetours-morocco.com/fers/viagra-generic-name.html of for. Get "drugstore" is chemicals. Doesn't http://augustasapartments.com/qhio/cialis-user-reviews products thinking easily metal the! Already "click here" But future glowing viagra canada online and moisture in Grade it.
breasts) in la casa Tripepi.
ingredients that my recipes are laden with.
cheeses are perfect for these.
– in fact, you can use your favourite cheese.


Between the ages of 13 and 17 I did a lot of babysitting...a lot. For a couple years it was to earn the money for a second week of summer camp and for the other years it was just to earn spending money for all the things your parents, rightly so, shouldn't pay for!
Those memories of long ago came rushing back as Anne's niece brought her daughter and son to be introduced to their great aunt. "Darling" doesn't even begin to describe them and the day we spent just watching them and playing with them.
Vacation days should be different from "normal" days they say--this one certainly qualified...in spades!


One of the best kitchen design tools available today is the Flemington Granite Kitchen Visualizer. It can often be difficult to visualize what your kitchen will look like after it is remodeled. Even more confusing is deciding which type of countertop to use, what edging to choose and which colors will look best in your kitchen. However, it is easy to get distracted with the many options for counters, floors and cabinets, so it is important to follow a few tips.
Professional designers start their projects by looking at one element and it is usually cabinets. One reason for this is that cabinets are often the most expensive part of any kitchen remodeling project and are the most permanent part of the room.
Although you may change paint color or appliances, your cabinets are usually not replaced until they have major problems. In addition, designers understand that design is about contrast so choosing cabinetry that is either light or dark can direct the theme for the rest of the kitchen.
The first step is to decide if you want dark or light cabinets. Although the current trend is for dark cabinets with lighter countertops. One of the reasons this is trending is that dark cabinets hide marks, drips and spilled items in between cabinet cleanings.
Using the virtual kitchen design tools in the Kitchen Visualizer, you can compare dark cabinets in cherry or coffee with light countertops to see how it could look in your kitchen. Once you have found the cabinet color that appeals to you, you can move on to the next element in your kitchen.
Flooring is the second most dominant color in your kitchen and it is also an element that is not changed often. You should resist the urge to go from choosing cabinets to countertops for these reasons. Keep in mind that you want flooring that contrasts the cabinets but complements them as well.
If the cabinets and flooring are too close to the same color, it could be overwhelming when people step into the room. Flemington Granite's kitchen planner tool can help you choose the flooring that best complements your cabinets. Change the cabinets and flooring until you find the combination that works best.
Now that you have decided on flooring and cabinets, you can move on to countertops. You don't want to choose countertops that look exactly like the cabinets or floors or the room will look too busy. However, you want to be sure that the kitchen feels harmonious. Countertops provide horizontal lines in the room while cabinetry gives you a vertical look that contrasts the counters.
Use the free kitchen design software from Flemington Granite that works well with the cabinets and flooring you chose. Keep in mind that countertops are replaced sooner than cabinets and flooring, but you want to choose a style that you can live with for a long time.
The Kitchen Visualizer tool allows you to design a kitchen in the same way that professional designers use. Many designers use virtual kitchen design software to demonstrate their own vision to clients. With Flemington Granite's kitchen planner tool, you can get an idea of what your kitchen will look like after the remodel.


Turn dog poo into a great compost!
Our Pet & Dog Poo Wormeries are a great way to turn pet and dog poo into great worm compost and a liquid fertiliser for your garden.
Because of the nature of the waste (faeces) we do not advise using the end compost in vegetable patches or in areas where young children would play - but it is an excellent conditioner to be dug into your garden or used as a bedding for shrubbery and flower beds.
Available in multiple colours with 3 or 4 tray options.


I got a hold of the owner of gt4-racing.eu he is willing to ship some bumpers and misc other parts he has in stock over here on a pallet. I would then have to drive to my nearest port and pick them up and then would ship them to you. Right now with 10 bumpers the price will be $250 each plus shipping from 68133. This price would obviously go down with more interest. I have this posted on celicatech and st162.net also to gauge interest.
You already know I'm in. Glad to see my brain child come to fruition!
Will it be possible to get some sets of the fenders in the group buy as well? I'd be interested in a set as well as a bumper and might know a couple more people interested in the fenders.
might be interested in some st165 stuff. think we can do those too?
I looked on the website I will do a st165 bumper with side moulds.
I will check on fender pricing I think it will be around $300, but he an definately ship those and st165 bumpers will be fine too, I'll see if we can get a bigger price break.
I would be interested in one of their engine bay under guards.
Have you actually used them? I've searched high and low and tried just about every website and dealership I could try and I've placed a order 3-4 times only to have the business email me and say they can't import the part.
Spectra1 wrote: I would be interested in one of their engine bay under guards.
Plus 1 for this part.
you actually cant order them from Toyota anymore there is 2 different part numbers for the rc bumper that are available in the US. and there are only 3 total in the states, but if you try to order them toyota will reject it unless your car came stock with it. Also you can't try a vin number from a car overseas, they will say go there to order it.
I know this because i tried ordering it, and that is what the dealer told me.


Leah Yael Levy is a mixed media artist, storyteller, printmaker and teacher based in Oakland, CA.
Originally from Israel, She first moved to New York City in 2002 to attend the Art Students League of New York, and later gained a BFA of Illustration from Parsons the New School for Design (2011). She recently received an MFA in Comics from California College of the Arts (2017). Her comics have been published by Forward Comix and The Nib.


text:
African swine fever background
African swine fever (ASF) is a viral disease that only domestic pigs and wild boar are susceptible to. The disease occurs primarily in Africa, more specifically in the countries south of the Sahara. From 2007, outbreaks were observed in Georgia / Eastern Europe and from 2014 also in the European Union.
The African swine fever virus
The African swine fever (ASF) virus is the sole member of the Asfarviridae family. The disease was first described in 1921, in Africa. It is the only DNA virus known that is able to infect arthropods (certain soft-bodied ticks of the genus Ornithodoros), as well as mammals.
Survival in the environment
The virus is able to survive for several days in the environment. However, this may be extended in the presence of protein (blood, meat) to weeks or months, and even a year. For example, the virus can remain infectious in dried hams (Serrano and similar) for up to 4-5 months. This may even rise to years in frozen meat.
The virus can be rendered inactive by heat treatment (at least 20 minutes at >60 °C, 70 minutes at >56 °C), pH <3.9 of >11.5 and responds well to most disinfectants.
The host: suids and ticks
The natural hosts for the virus are suids. This historically refers in particular to warthogs (Phacochoerus spp.), but also for example to bushpigs (Potamochoerus spp.) and to the giant forest hog (Hylochoerus meinertzhageni), all of which are endemic to Africa. Domestic pigs and the wild boar (Sus scrofa) are also susceptible to this virus.
Infection of Soft-Bodied Ticks
Certain ticks can also become infected with the virus. These are the soft-bodied ticks of the Ornithodoros genus. These ticks occur only in tropical and subtropical regions. For example, in Europe only around the Mediterranean Sea.
Harmless to Humans
Other animals are not susceptible under natural conditions. The virus is also completely harmless to humans.
Infected pigs
Soft-bodied tick; photo Joseph Sarr, ISRA
Where does the disease occur?
African swine fever, as the name suggests, occurs primarily in Africa, more specifically in the countries south of the Sahara.
Between 1957 and the mid-1990s, the disease spread to Europe, and there were outbreaks in the 1970s in the Caribbean and Brazil. By the mid-1990s, the virus had been eradicated everywhere outside of Africa, with the exception of the Italian island of Sardinia.
Recorded in Europe again in 2007
In 2007 the virus turned up in Georgia, in the Caucasus region. From there it spread to surrounding countries, including Russia. Over subsequent years, the virus spread across virtually the whole of Western Russia up to the border with Finland. In 2012, an outbreak was recorded in Ukraine, followed by one in Belarus in 2013. In 2014, the first cases were observed in the European Union, in Lithuania and Poland. The virus then soon spread to Latvia and Estonia as well. In 2016, it spread to Moldavia, and in 2017 to the Czech Republic and Romania. In Poland, the virus moved to the Warsaw region.
Outbreaks in 2018
From 2018, outbreaks have also been registered in Hungary. And in Romania, the biggest pig farm in the country was hit by the virus. In Poland the virus advanced further to the area around Warsaw. In September 2018 the virus made a big leap and hundreds of wild boars in our neighbouring country became infected. There are also major outbreaks in China and Mongolia, where around 100 pig farms have been affected by the disease since August 2018.
Check the website OIE - World Organisation for Animal Health for the latest information.
How do suids become infected?
Pigs and wild boars can become infected in the following ways. For an explanation read on below.
Soft-bodied ticks, warthogs and the cycle in Africa
In Africa, the reservoir of the African swine fever virus is found in the wild. The warthog and the soft-bodied tick (mainly of the species Ornithodoros moubata) maintain the virus cycle there. Soft-bodied ticks feed on infected warthogs, propagate the virus and pass it on to another warthog at their next blood meal. As warthogs and ticks cohabit closely in warthog burrows, this is an efficient method of spreading the virus. Direct mutual contact between warthogs probably plays a more limited or even no role in spreading the virus.
Distribution among wild boar
Wild boar, as known in Eurasia, can also become infected with the African swine fever virus. Their role can certainly not be compared to that of the wild African suids. In wild boar, the disease is usually acute, and virtually all infected wild boar die. Over recent years we have seen the virus being found increasingly in wild boar, and it seems that under certain circumstances, the virus can continue to circulate among wild boar over the longer term or even become endemic.
Infection of pigs and wild boar may occur via direct contact between an infected animal and a susceptible (as yet uninfected) animal. This ensures distribution within an infected pigsty, for example. Contact between wild boars and pigs kept outside is also an example of this.
Contaminated Feed
All outbreaks outside Africa probably started with feeding kitchen waste (swill) from for example ships or planes arriving from Africa.
The virus can easily end up in the food chain through the slaughter and processing of infected pigs. This is not unusual, particularly in small-scale pig-keeping, where people keep just one or a couple of pigs. The virus can survive for a long time in products of this kind. When these products are moved, the virus can be spread over large distances, with the result that it is extremely difficult to suppress. Products like dried sausage or salami can for example be a source of infection, if fed to pigs, taken to a pig farm or left in an environment where they can be reached by wild boars. The sources may be tourists, workers/employers or truck drivers on long routes, alternatively hunters carrying trophies or wild boar products.
Contact with Carcasses
Contact with carcasses or remains left from a hunter gutting a wild boar may result in further infections in wild boar. In addition, as stated above, food remains left in the environment can infect wild boar.
Contaminated Materials
Contaminated materials, such as tools, boots and also stable bedding may cause infection in pigs. The quantitative risk of the virus ending up in pigs via this route is not entirely clear at present. Livestock transport vehicles in which infected pigs are being moved, also pose a risk.
Present long term in surviving pigs and wild boar
Pigs that survive an initial infection become carriers of the virus. They can carry the virus for several months. These pigs are much less infectious than pigs in the disease's acute phase. Nevertheless, they may have an important role in the epidemiology, because they ensure that the virus can remain latent over the longer term.
Do you have a question about African swine fever? Ask our expert: dr.ir. TCW (Tosca) Ploegaert
Back to the homepage of African swine fever
Wageningen Bioveterinary Research (WBVR)
Top level veterinary and biomedical research for animal and public health
Facilities & research
瓦赫宁根大学及研究中心
研究及成果
专业与服务
专业与服务 expertiseservices
Follow us on: Twitter Facebook
Prospective BSc Students
Prospective MSc Students
Climate and water
Biobased economy
Logistics and trade

redpajama_set_name: RedPajamaCommonCrawl


text:
How millennials are changing the face of giving
Are millennials more generous than generations past? Nonprofits sure hope so. December 1, 2017 | by Sunny Montgomery | Copyright © 2022
Millennials are changing the face of giving
In the early 1970s, baby boomers witnessed the dawn of a new Chattanooga. Thanks to environmental efforts, the industrial smog began to lift from the city deemed the most polluted in America just a few years prior. The surrounding mountains glowed green once again, giving locals a new sense of community pride and, consequently, reasons to invest in their city.
Maeghan Jones
"Millennials weren't part of that turnaround story," says Maeghan Jones, CEO and president of the Community Foundation of Greater Chattanooga, a publicly supported charity with the goal of building permanent, sustainable funds for the public benefit. Millennials, Jones says, value the city for different reasons, such as its natural beauty, cutting-edge technology and potential for startup companies.
Both locally and nationally, the way millennials see the world is unique, and that is changing the way nonprofits think about fundraising. With more than 75 million millennials alive today, those born between 1981 and 1997 make up the largest living generation.
If organizations can connect to these potential donors, they stand to gain more than ever before — $30 trillion, to be exact, the estimated amount of money that up-and-coming generation will inherit from their baby boomer parents over the next 40-50 years.
Both locally and nationally, the way millennials see the world is unique, and that is changing the way nonprofits think about fundraising. With more than 75 million millennials alive today, those born between 1981 and 1997 make up the largest living generation. If organizations can connect to these potential donors, they stand to gain more than ever before — $30 trillion, to be exact, the estimated amount of money that up-and-coming generation will inherit from their baby boomer parents over the next 40-50 years.
"That transfer of wealth is coloring everything ," Jones says.
In Hamilton County alone, she says, that inheritance is an estimated $9.2 billion within the next 10 years. "Imagine if just 5 percent of that went to create permanent charitable assets," says Jones. "That's $460 million for education and the arts — money to enhance the community."
But in order to tap into that potential, organizations must capture the attention of the increasingly busy millennial.
Patrick Long
"Donor behavior is shifting," says Patrick Long, vice president of strategy and engagement for United Way of Greater Chattanooga, an organization that supports community issues, such as education, primarily by issuing grants to social service organizations. "General email blasts are not good enough this day and age. You have to fit your organization into [donors'] lives in a way that is meaningful to them, and through a medium that they want to use."
United Way Emerging Leaders attend the Great Conversations event, at which business men and women under the age of 40 came together to network.
But more than just opening new channels of communication via email, text or social media, for example, to reach millennials, organizations must understand their psychology. What influences them to give, or connects them to a cause? How do young donors derive meaning from their generosity?
"We see a lot of shared values across all our donors, but they manifest differently," Jones says. "No generation is monolithic, but there are trends."
To better understand those trends, Jones and Long examine the three big ways that millennials are philanthropically different than past generations.
Purposity launch party, hosted at Whiteboard.
No. 1: Millennials are peer-influenced.
There was a time when a person's loyalty to a charity was determined by his or her employer, Long says, but millennials won't sign a campaign pledge form just because it was left in their company mailbox. Rather, their loyalty is more influenced by their peers, likely contributing to the rise of "giving circles."
Giving circles have been steadily growing in popularity since the early 2000s. They are a new form of philanthropy similar to crowdfunding, but more involved. In giving circles, a group of individuals pool their dollars and make collective decisions about where to donate that money. According to a report published by Connected to Give, a California-based philanthropic research organization, nearly half of those donors nationwide are under the age of 40.
Many organizations believe it is the networking opportunities and the relatively small personal monetary investment that attract millennials to giving circles.
Chattanooga's UnFoundation, established to help support startup programs with micro-grants, was co-founded by Teal Thibaud, Ben Garrison, Tavis Salazar, Bijan Dhanani and Joda Thongnopnua — all of whom are under the age of 40. Its members attend monthly meetings in addition to contributing $100 per month.
Similarly, Chattanooga's Sankofa Fund for Civic Engagement, which funds projects to improve local communities of color, is predominantly comprised of individuals age 40 or younger. Members are required to contribute at least $600 per year.
"Giving circles are not just the home for millennial donors, but for people who have traditionally been excluded from philanthropy," Jones notes, explaining that historically, the face of giving tended to be white and male. "Women in philanthropy is another trend we're pretty excited about," she adds.
In fact, women are positioned to play a critical role in fundraising's future. A 2011 study by Bank of America Merrill Lynch found that within 90 percent of high-income households, women are now the sole decision maker or at least an equal partner in deciding which causes to support. Moreover, studies have consistently found that women give more than men.
In Chattanooga, women-run giving circles such as the Nightingale Network hope to strengthen this trend by connecting female donors and raising funds for the purpose of improving the lives of women and girls in the region.
Purposity is an online platform that connects millennial givers to specific causes.
No. 2: Millennials are not brand-loyal.
I think back to when my parents gave a donation; they were really kind of trusting and just going on faith that the organization was doing good work," says Long, 31. Then, the advent of the internet allowed prospective donors to pull an organization's tax forms and critically examine its financial impact. Moreover, Long says, "millennials grew up in a world of sophisticated marketing," which made them both savvy and skeptical. "What an organization says is not necessarily trustworthy to the millennial," Long says.
Consequently, millennials rely more on their peers for giving guidance, but also, they are loyal to causes — not organizations. In the near future, Long says United Way hopes to appeal to these up-and-coming donors through cause-oriented mini-campaigns. Rather than asking for a one-time donation or a one-time payroll deduction, the organization will send out a series of emails asking for "$15 to end homelessness" or "$15 to end domestic violence," for example.
"It's lower risk for the donor. They can test the water with that small donation," Long says.
Because in addition to being cause-loyal, millennials are impact-driven. They want their contribution to yield quick, tangible results.
"Purposity is a powerful and very present example of the way these influences have transformed approaches to generosity here in our own community," Jones says, referencing one of the Community Foundation's millennial-centric initiatives.
Developed by Whiteboard, a web development agency in Chattanooga, Purposity is an online platform that connects millennial givers to specific causes. Once a week, members are texted a list of local needs — a winter coat for a child, for example. Then, members select which need they want to meet and make that purchase, a process that takes under two minutes.
The platform, which launched in January, currently has 1,068 members in Chattanooga.
"Thirty-six percent of [Purposity] users are between 18 and 34, and they've met 40 percent of the [650] needs," says Jones. "They get a text; they meet a need; they know directly that it has this impact. It's instant gratification."
No. 3: Millennials want to feel unique.
The concept of donor-centered fundraising arose in the early 2000s, Long says. Rather than focusing on donations, this new approach focuses on the donor. "How do you make the donor the hero?" Long explains. One way, he suggests, is a simple shift in language. "In the opening paragraph of any letter, replace 'we' with 'you,'" he says.
But beyond just monetary donations, millennials' desire to feel unique changes how they give their time, too — leading to another relatively new concept: skills-based volunteerism.
"Millennials don't want to come into a food bank and just pack boxes," Jones says. "If they work for a logistics firm, they want to come in and help with the supply chain."
Volunteers who integrate their professions with their philanthropy tend to be engaged on a more personal level, she adds. Thus, those volunteers are more likely to leave planned gifts, a type of contribution commonly made through a will or trust after the donor has passed away.
From giving circles to apps to legacy gifts, the success of the modern-day philanthropic model depends on presenting young donors with more ways to connect. But while millennials are changing the way the world talks about fundraising, the essence of philanthropy transcends generations.
"It is about compassion. It is a love of humankind," Jones says.
Give thanks to area music venues that give local artists a voice
Here are ways to get drinks delivered to your door
Chatter Sections
People Scene
Subscribe to Chatter
Jennifer Bardoner
editor@chatterchattanooga.com


text:
Written by on December 17, 2019
Grammy Awards Live Stream are a single quarter with the major 4 awards ceremonies to occur on a yearly foundation, together with the other three staying the Emmys for Television set, the Academy Awards for motion pictures, as well as the Tony Awards for theater. The Grammys concern new music, and so are award to your top rated musicians, songwriters, experts and organization people today in that earth.
The Grammys started out in 1958 and received their identify within the identify of your initial file participant, which was identified as a gramophone. The way it works is nominations are sent by any person linked to tunes, regardless of whether it be musicians, studios, or other intrigued events. These bids are despatched for the National Academy of Recording Arts and Sciences, who filter by way of all of the submissions and determine the best 5 names in just about every group. Then people names are sent out to many of the customers on the Academy, who then are requested to vote for someone from the standard classes as well as focus on those who are within their personal specialties or style. The 4 general places are: Album of your 12 months; File from the Yr; Tune with the Yr; and Greatest New Artist.
The Grammys are deemed the top in the new music awards, nonetheless they also generate one of the most controversy. Some have identified as the awards elitist or illogical since a lot of music or albums that are nominated did not offer properly, and occasionally honor names that numerous persons haven't heard about. That works effectively for specialized awards although not as well when these artists clearly show up in popular categories. There's also criticism that not ample attention is compensated to many of those people who are regarded as for that awards, which was highlighted most especially in 1990 when Milli Vanilli gained the Grammy for Best New Artists in advance of getting exposed as not becoming the artists that carried out on their document or in live performance.
A further controversy is usually that, not like other awards, not all tunes or artists are identified by individuals who vote in these categories. Whereas the Emmys and Academy Awards ship goods to its voters for them to peruse, and voters for that Tony's needs to have found each and every demonstrate to vote, the Grammy's you should not give products and rely on their voters to try and do all this for them selves, which every person is aware of they don't do. Consequently, some options could possibly be centered on an in general physique of labor for an artist as an alternative to whether it was the very best for that individual yr.
0 Responses to The Grammys


Can you be more specific? What tape are you trying to access? Unlike disk drives, Commodore systems like the Vic-20 access cassette drives very easily. LOAD"program_name" loads a program from cassette, then RUN runs it. SAVE"program_name" let's you save to tape.
Easiest way to test it is to write a simple two or three line program in BASIC, do a SAVE"program_name" (follow instructions on screen). When finished and you stop the tape, rewind it. Do a LOAD"what_you_saved_it_as". Do a LIST or RUN.
Commodore computers access the disk drive very easily too, just add a device number after the LOAD/SAVE command. Some models (not VIC-20 though) came without tape support, and on those the default device to LOAD was disk drive #8.
If a test program is possible to load back, perhaps the other tape really doesn't contain VIC-20 software, it was recorded at a different azimuth angle or the tape has lost its data.
Obviously you only add a drive # or drive #,1 to access a file on disk, but doing things like a directory listing Load"$",8 or initializing (formatting) a disk are not intuitive. My point was, the tape drive access is about as straightforward as you can get, versus a disk where you need to know specific parameters for access and other functions. Utilities like the ones on cartridges helped with these too.
lol, that was all, I am such a commodore newb.
True, formatting a disk is a bit more complex than just popping in an unused tape. Directory is not really possible with a tape anyway, so you'd have to write down what the tape contains.
I used an old tape and recorded the "fill screen with characters" program, and it saved, and accessed...Seems to work very well.


text:
Show Us Your Nindies – Don't call it a throwback
Powah Dunk | June 23, 2020 | Otaku Review | No Comments
This article was originally due to go live in June 2019!. While sorting through other files I came across and it decided to give it a little tidy and publish it. While not ideal unfortunately some games do slip through the cracks being a small team with limited time! I hope you enjoy this article and the games discussed – Andy
Mechstermination Force
Mechstermination Force is the latest title from the developers of the fantastic Gunman Clive series, once again they are tackling the 2D platformer shooter genre but who better to do so than this team?.
Mechstermination Force puts you in the milliatry boots of one of four characters before it thrust you into the war on Mechs. There is little bits of narrative in the hub area of the game and honestly more dialogue than I expected from this title but don't expect War & Peace, it's simple, it's to the point and it certainly isn't a title you are playing for deep lore!.
Visually Mechstermination Force is a fantastic looking game with a rather uninspired art style which is the main downfall of this title. It isn't a terrible looking game and kind of hits it's mark in a 90s Sci-Fi, action flick style. The backgrounds are throw away and the playable characters are as you'd expect from a game called Mechstermination Force, even the Mechs themselves are varied enough and the sheer size of them is enough to make them stand out. The only issue lies within the art style not really having it's own identity, unlike the teams previous titles or other Boss Rush games available.
The gameplay is where this title truly hooks you if you weren't already sold on the idea of 90s platform shooting against Giant Mechs, to put it simply the team behind this title has fused the old classic Contra with the scale and puzzle elements of Shadow Of The Collosus to create an insanely addictive boss rush game with tight controls, brilliant multiplayer and enough insentive to replay the bosses over and over again.
Yes this game has no "Levels" in a traditional sense, it skips you straight to the challenge of the boss from the get go. The bosses are all incredibly varied, large and hit like the giant trucks they are. Throughout the boss battle you'll have to figure out how to expose the cores of the mech, then how to get close enough to give them a hefty melee whack, this is usually with comically oversized items such as guitars and other items you wouldn't expect.
As you progess through the battles you earn coins which you can spend in the shop to gain access to new weaponry, extra health and other items in which to help you through the games 14 boss battles.
I know 14 boss battles doesn't sound alot but you will be restarting them time and time again just to learn the patterns and solve the puzzles of just how to bring these Mechs down, every battle is unique and aims to keep you on your toes and fully engaged until the score screen drops on your victory.
There is also a simulation mode in the game which allows you to return to any boss fight previous, earn more coins and try and get all the gold starts for the fight by completing them in the most efficient time, with no damage taken. It's a hefty task and one you'll enjoy trying to finish with this games tight controls and addictive gameplay.
I can't recommend this title enough, it's got such a SNES vibe to it with more current graphics, it scratches any boss rush itch, very fairly priced and is an incredible time sink, be it just spending 10 minutes or so to try and figure out a boss or losing hours trying to get all those gold stars. Mechstermination Force is an asset to the genre, to the Nintendo Switch and to the developers ever-expanding legacy, you shouldn't sleep on this title, especially if you are a fan of the genre or just fancy trying something new.
Giga Wrecker Alt
Giga Wrecker Alt is a new title from a little known team called Game Freak (yes the Pokemon people!), originally released on Steam a few years ago, Alt is an expanded version with several additions for the Nintendo Switch.
Giga Wrecker Alt is a physics based "Metroidvania" which combines platforming, puzzles, exploration and debris management.
Giga Wrecker Alt has an interesting anime based art style with a smattering of style usually seen in Vanillaware titles, on the surface it looks quite nice but the in game character models are quite stiff and unlike the comparison it doesn't quite pull the effect off in a satisfactory way.
The market as of the last few years has an abundance of "Metroidvania" titles and Giga Wrecker Alt is another, it's main hook is you use debris around the world to fight and solve puzzles. The combat is satisfying, you have to have a certain size of debris to destroy an enemy and you can add them to your ever growing ball of death. Alongside the combat the debris are also used for many physic based puzzles, if you struggle with these then this version of Giga Wrecker includes hints to help you get to where you are going.
The world is a confusing mess of similar looking areas with no distinct character to them, it's this which really knocks the game for me, it feels like you're exploring the same area for the entire game, I lost my compulsion to explore a few hours in because I didn't feel really rewarded for doing so. The map also isn't the most intuitive thing ever, especially when compared to its kin within the genre.
My final gripe with this game came from playing in Handheld mode. In order to show the whole area the game zooms out and it makes it very hard to see the character and any of the detailing, the bigger the area, the bigger the problem.
Giga Wreckers Alt is an interesting title if not flawed in several key areas. The story starts off interesting but dries out long before you get to where you are going, much like the gameplay experience. It's not a bad game by any stretch it's just difficult to recommend in the current market of the genre.
Powah Dunk
Straight from the streets of SouthTown, all Dunks Powah'd and ready to Bust A Wolf. Catch me on Twitch/YouTube.
Latest posts by Powah Dunk (see all)
Wings of Bluestar (Switch) - January 20, 2023
Omega Bot (Switch) - January 18, 2023
Onion Assault (Switch) - January 17, 2023
Tags:Reviews
DynaMarisa 3D (PC)
Trinity Souls of Zill O'll (PlayStation 3)
Ikki Tousen: Shining Dragon (Ps2)


SMTP is used to send and receive email. It is sometimes paired with IMAP or POP3 (for example, by a user-level application), which handles the retrieval of messages, while SMTP primarily sends messages to a server for forwarding. SMTP can both send and receive mail, but it's bad at queuing incoming messages, hence the common delegation to other protocols. Proprietary systems like Gmail have their own mail transfer protocols when using their own servers, but they still use good old SMTP to email beyond that.
SMTP is an asymmetrical protocol, meaning that there are many clients interacting with one server, using a basic model popular in the 1980s which is now mostly defunct outside of email protocols. SMTP runs on TCP/IP and listens on port 25.
What's the Difference Between SMTP and Extended SMTP?
ESMTP is a new version of SMTP with added functionality. Released in 1995, it can send multimedia files. Now your aunt can attach videos of her cat to her weekly email update. Thanks, Internet.


Grade: 7 yrs. to 14 yrs.
Using our progressional curriculum and focusing on the whole player, we teach your child the skills they need for both on and off the court to be a better athlete. Our sport specific staff will focus on respect, teamwork and responsibility as they will start each day with a 'skill of the day', and progress into drills and games making for an unforgettable camp experience for your young athlete. After a week of passing, shooting, dribbling and rebounding you will see why this is one of our most popular programs.
This camp is perfect for the beginning to intermediate basketball player.


My name is Donda Zimmer and this is my story. I am a military wife and the proud mother of three beautiful children. In 1997, I gave birth to my first child and nearly passed away due to complications with the birth and congestive heart failure. After the birth, my exercise regimen was very limited and as a result I continued to gain weight throughout the years. During my second and third pregnancy, I had to wear a heart monitor and suffered many setbacks over this time. It was after giving birth to my third child that my fitness journey really took off!
In 2003, my husband was selected for an unaccompanied year long deployment. I had just given birth to my third child and gained more than 30 pounds and knew it was time to make a change. I began running and altering my eating habits. Within the first two months I had lost the 30 pounds and started packing on the muscle. This was the start of my profession and my ministry.
After my husband returned from his tour in Korea, we were re-assigned to Anchorage, Alaska. This is where I decided to enter my first Fitness and Figure competition. I placed third in my division and developed the discipline, motivation, and dedication to compete and live a healthy lifestyle. Every day I trained harder and harder. I was determined to become bigger and better. Two years later we received a new assignment to Colorado Springs, Colorado. I spent my time here taking advantage of the high altitude improving my overall cardiovascular endurance while increasing my muscle mass. Two short years later, we had an assignment to San Antonio and this is where I encountered life. I received news that I would have to undergo multiple life changing surgeries, was diagnosed with fibromyalgia, and witnessed my mother lose a long battle with cancer.
During this difficult time of recovery and great loss, the Lord spoke to me and equipped my heart to go and serve others by helping them change their lifestyle and educating them on healthy living and that's exactly what I did! I have now completed the National Academy of Sports Medicine Personal Training Certification, Fitness Nutrition Specialist certification, and Weight Loss Specialist Certification. I own my own business and recently jumped back into the world of competition. Last November, I competed after undergoing going two foot surgeries in the "Lackland Classic" and placed second in the Women's Physique division.
This is my testimony! Training Champions for Christ!


Smash your enemies in this tweet battle to prove who is the strongest. Tap as fast as you can to charge your tweet and unleash your power.
Win matches to get trophies and move up positions to be the best. Play survivor mode to win gold and buy accesories or paint your tweet.
Play in training mode to see how many taps you can do.


text:
Career Center November 9, 2017
Beyond Accounting Firms: Should You Work for a Public or Private Company?
by Grant Hutchinson
We all know the story. It follows a classic 3-act structure:
Act I — A young student discovers accounting and falls in love with the potential it will have for his career. Our protagonist believes that a job with an accounting firm will lead to great success, and that he will follow the steps of many who have come before and went on to great prosperity as controllers and CFOs in public companies.
Act II — Discouraged by public accounting villains and obstacles, our hero presses on. His superiors insist that his career will take off as long as he stays until manager. So he persists, despite enticing offers from prestigious public companies.
Act III — After a final confrontation with a partner about an obscure portion of some recent derivative accounting guidance, our hero storms out of the office, never to look back. He literally walks into a public company CFO on the street whose controller just moved to Wichita to take care of his aging mother. "You wouldn't happen to know of anyone with 8-10 years of public accounting experience and excellent communication skills who's looking for a new job, would you?" the CFO asks.
*Curtain*
Hollywood has yet to capture this gripping tale of man-meets-numbers on film, but it's only a matter of time.
Isn't it time for a new story?
But what if the story were to change a bit? What if the accountant in Act I only thought he wanted a career with a public company, but in Act III discovered a private company accounting job that made him just as happy?
As the complexity of our economy grows, the once sharp lines between public and private accounting are beginning to blur. There are now a number of private organizations that can provide just as many benefits as a public company.
A private company could be a better path
One example of a private company that could sway our hypothetical accountant away from the standard career path is found within health insurance giant BlueCross BlueShield. Part of the reason for this is that BCBS's unique corporate structure allows it to offer advantages to its financial professionals that other private businesses generally can't.
Unlike traditional enterprises, BCBS operates as more of an association of licensed sub-companies, each of which has some leeway in corporate structure.
Some members of the BCBS Association, including BlueCross BlueShield of Tennessee, have chosen to conduct business as a non-profit. While BlueCross BlueShield of Tennessee is licensed to offer certain products and services under the corporate brand, the organization runs as an independent, locally governed private company.
Accountants at BlueCross BlueShield of Tennessee benefit from the enormous financial, technological, and informational resources of the BCBS network. And as employees of a non-profit, non-public company, they get the job satisfaction that comes from working to improve the health of people, not just the health of the company. It's like the best parts of public and private accounting merging into one hybrid organism of awesomeness.
Act IV: A New Hope
Maybe it's time for the story to change. As the profession continues to mature, building a successful, fulfilling career within private organizations is becoming an increasingly viable path.
Your public-to-private or even private-to-private transition can start right now. If you're in the Chattanooga area or are interested in relocating there, BlueCross BlueShield of Tennessee is currently hiring.
About BlueCross BlueShield of Tennessee
As Tennessee's largest health benefit plan company, we've been helping Tennesseans find their own unique paths to good health for over 65 years. More than that, we're your neighbors and friends – fellow Tennesseans with deep roots of caring tradition, a focused approach to physical, financial and community good health for today, and a bright outlook for an even healthier tomorrow.
There are currently no vacancies.
Tags: BlueCross BlueShield of Tennessee, GC Gigs, sponsored content
Career Center December 21, 2021
We Did It Guys, We Found the World's Most Insufferable Accountant
Once again, r/linkedinlunatics comes through with the gold: Either this guy is really, really serious about his job OR — and more likely — just really bad at figuring out how to get 16 hours of work done in 10. If Reddit existed 20 years ago, the comment section on that post would likely be […]


text:
\section{Introduction}

Given a learning algorithm, evaluating the learning performance for any arbitrary input dataset traditionally requires training with the dataset and then calculating the performance of the trained model. The computational complexity of this evaluation process is often dominated by the training part, which could be very expensive. On the other hand, a variety of applications could potentially benefit from an efficient process of estimating the learning performance. Examples include noisy or adversarial data removal, data summarization \citep{wang2021unified}, and active learning~\citep{wang2021oneround}, which can all be done with the same recipe -- evaluating the performance of many subsets of data points and only retraining the subset that achieves the highest learning performance.

\newcommand{0.35}{0.35}
\begin{wrapfigure}{R}{0.35\linewidth}
    \centering
    \includegraphics[width=0.35\textwidth]{images/mnist-ins.png}
    \caption{Figure from \cite{wang2021oneround}: An illustration of data utility functions for a logistic regression model trained on MNIST dataset. The three lines are each generated through randomly sample and add a data point to the training set each time, and record the logistic regression accuracy with the increase of training data size. }
    \label{fig:approach-submodularity}
\end{wrapfigure}

One natural idea of estimating learning performance \emph{without training} is to learn the mapping from a dataset to the resulting learning performance. We will refer to the mapping as the \emph{data utility function} hereinafter. Each training sample for learning a data utility function consists of a set of data points and the corresponding learning performance. While a learned data utility function allows efficient learning performance prediction for any given input dataset, the training samples for learning data utility function are expensive to construct, as each sample requires training on a dataset. Hence, there is an urgent need to understand the sample efficiency of the learning problem. Recent work shows promising empirical results that for many common learning algorithms, the data utility functions can be accurately learned with a relatively small amount of samples \citep{wang2021oneround}. The authors conjecture that the observed sample efficiency might be caused by the approximate submodularity of data utility functions (see Figure~\ref{fig:approach-submodularity}); specifically, most times, an extra training data contributes less to learning performance as the training size increases. There is a rich line of work that studies the sample complexity bound for submodular functions~\citep{balcan2011learning, gupta2013privately, feldman2016optimal, feldman2020tight}. However, there still lacks an understanding of (1) \emph{how to rigorously characterize functions with ``approximate'' submodularity} and (2) \emph{how the approximate submodularity affects learnability}.

In this paper, we propose a relaxed submodularity condition that can well describe data utility functions of common learning algorithms. We show that functions satisfying the ``relaxed submodularity'' condition is subsumed by a larger function class called $(a, b)$-\emph{self-bounding} functions, which enjoy the dimension-free concentration bound \citep{boucheron2000sharp}. Leveraging the concentration bound, we show that these functions can be learned with constant-factor approximation error under the PMAC learning framework. Notably, our result applies to general self-bounding functions, which may be of independent interest.  Furthermore, we show that $(a, b)$-self-bounding functions can be approximated by low-degree polynomials and then leverage this structural result to show that they can learned with $2^{\mathcal{\tilde{O}}((a+b)^2/\epsilon^2)}\log n$ samples under the PAC learning framework. Both PMAC and PAC framework are widely used in analyzing the learnability of real-valued functions.

To showcase the usefulness of data utility function learning, we study its application to data valuation as a concrete example. The goal of data valuation is to quantify the usefulness of each data source for downstream analysis tasks. Data valuation has gained a lot of attention recently because it can be used to inform the implementation of recent policies aimed at enabling individuals control over how their data is used and monetized by third-parties~\citep{voigt2017eu}. Moreover, the characterization of data value enables users to filter out poor quality data and to identify data that are important to collect in the future~\citep{ghorbani2019data, jia2019towards, jia2019efficient, jia2019scalability, wang2020principled}. Existing work leverages concepts from cooperative game theory, such as the Shapley value and least cores, as a fair notion for data value.
However, computing or even approximating these data value notions requires evaluating the learning performance of many different combinations of data sources, which could be very computationally expensive. We propose to learn a data utility function and use it to predict the learning performance of a subset of data sources without retraining. We conduct extensive experiments and show that data utility learning can significantly improve the approximation accuracy of Shapley value and least cores. Particularly, our approach can be extended to many other cooperative games where the characteristic functions are close to submodular and the evaluation is expensive, such as bidders' valuation functions in combinatorial auctions \citep{lehmann2006combinatorial}.

\section{Related Work}

Data utility learning was first proposed in \cite{wang2021oneround}, where the trained data utility models are used for active learning. \cite{wang2021oneround} observe that most of the common data utility functions can be learned efficiently with relatively small amount of samples.
They conjecture that the learnability of data utility functions is due to their ``approximate'' submodularity property. We note that the term ``approximately submodularity'' has different definitions in the literature \citep{horel2016maximization, hassidim2018optimization, das2018approximate, chierichetti2020on}.
However, these definitions do not consider special properties of data utility function observed empirically; e.g., the deviation from submodularity may depend on the sizes difference between two datasets.
More importantly, no existing works have studied the learnability of approximate submodular functions. This paper presents a rigorous characterization and learnability result for the ``relaxed'' submodularity functions that can well-describe data utility functions.  \cite{chierichetti2015approximate} and \cite{feige2020approximate} present algorithms for learning the closest modular function via queries, which is orthogonal to our focus.

Self-bounding functions were first introduced by \cite{boucheron2000sharp}. \cite{mcdiarmid2006concentration} further refine and introduce a more general notion of $(a, b)$-self-bounding functions. The most notable property of self-bounding functions is the dimension-free concentration bound, derived by the \emph{entropy method} \citep{boucheron2000sharp,boucheron2003concentration,mcdiarmid2006concentration, boucheron2009concentration}.
Particularly, submodular functions are special instances of self-bounding functions.
Similar concentration bound for submodular functions were independently derived in \cite{balcan2011learning} using Talagrand's inequality, unaware of the connection with self-bounding functions.
Our paper enriches the learnability study of $(a, b)$-self-bounding functions and explores the novel application of $(a, b)$-self-bounding functions to characterizing data utility functions.

Game-theoretic formulations of data valuation have become popular in recent years.
Particularly, Shapley value has been widely used as a data value notion~\citep{ghorbani2019data, jia2019towards, jia2019efficient, jia2019scalability, wang2020principled}, as it uniquely satisfies a set of desirable properties. Recently, \cite{yan2020ifyoulike} propose to use the Least core as an alternative of Shapley value for data valuation. However, exact computation of Shapley and Least core values are NP-hard under most of the settings, which limits their applicability in real-world data valuation applications even at scale of hundreds of data points.
Several heuristics, such as TMC-Shapley \citep{ghorbani2019data}, G-Shapley \citep{ghorbani2019data}, KNN-Shapley \citep{jia2019efficient}, have been proposed to approximate the Shapley value. Despite their computational advantage, they are biased in nature, limiting their applicability to sensitive applications such as designing monetary rewards for data sharing or assigning responsibility for ML decisions.
On the other hand, unbiased estimators of the Shapley value such as Permutation Sampling \citep{maleki2015addressing} and Group Testing \citep{jia2019towards} still require a large number of learning performance evaluations for any descent approximation accuracy.

The technique proposed in this paper is not another approximation heuristic for the Shapley value or the Least core. Rather, it is a natural way to improve the unbiased Shapley or Least core estimation, and it is compatible with any approximation heuristics that require a significant amount of data utility samples, such as permutation sampling \citep{maleki2015addressing}.

A similar idea of boosting the Shapley value estimation through learning the characteristic functions has been explored in cooperative game abstraction (CGA) \cite{yan2020evaluating}. However, their technique may not be directly applicable to data valuation due to the several issues. We provide a detailed comparison between CGA and our technique for data valuation in Section \ref{sec:dul-to-datavaluation}.

\section{Technical Preliminaries}
Since data utility functions are set functions, we start by reviewing important classes of set functions relevant to describing data utility functions.

Set functions defined on the power set of $N = \{1, \ldots, n\}$ can be equivalently viewed as functions defined on the binary hypercube $\{0, 1\}^n$.

\begin{definition}
A set function $f: 2^N \rightarrow \mathbb{R}$ is
\begin{packeditemize}
    \item \emph{monotone}, if $f(A) \le f(B)$ for all $A \subseteq B \subseteq N$.
    \item \emph{submodular}, if $f(A \cup B) + f(A \cap B) \le f(A) + f(B)$ for all $A, B \subseteq N$.
\end{packeditemize}
\end{definition}
Note that a submodular function is not necessarily monotone. An equivalent definition of submodularity is \emph{diminishing returns}: for every $S \subseteq T \subseteq N$, and every $j \in N \setminus T$, a submodular function $f$ satisfies
\begin{equation}
f(T \cup \{ j \} ) - f(T) \le f(S \cup \{ j \})-f(S)
\label{eq:submodularity}
\end{equation}

Next, we introduce \emph{$(a, b)$-self-bounding functions.} Let $e_i$ denote the one-hot encoding of $i$, i.e., $e_i$ is a vector of size $n$ with zeros on all but the $i$-th dimension. Let $\oplus$ denote the XOR operation.
\begin{definition}[\cite{mcdiarmid2006concentration}]
For a function $f:\{0,1\}^{n} \rightarrow \mathbb{R}$ and any $x \in\{0,1\}^{n}$, let $\min _{x_{i}} f(x)=\min \left\{f(x), f\left(x \oplus e_{i}\right)\right\}$.
Then $f$ is $(a, b)$-self-bounding, if for every $x \in\{0,1\}^{n}$ and $i \in[n]$,
$$
\begin{aligned}
f(x)-\min _{x_{i}} f(x) & \leq 1 \\
\sum_{i=1}^{n}\left(f(x)-\min _{x_{i}} f(x)\right) & \leq a f(x)+b .
\end{aligned}
$$
\end{definition}

Self-bounding functions is the most general class of functions currently known to enjoy strong ``dimension-free'' (i.e. independent of $n$) concentration bounds.

There are two commonly used analysis models for learning real-valued functions.
One is the generalization of Valiant's PAC learning model to real-valued functions \citep{valiant1984theory}.
Specifically, for any unknown function $f$ and target error $\varepsilon$, with non-negligible probability a PAC learner should output a hypothesis $h$ such that $\mathbb{E}_{x \sim \mathcal{P}}[|f(x)-h(x)|] \le \varepsilon$.
However, this does not differentiate between the case of having low error on most of the distribution and high error on just a few points, versus the case of moderate error everywhere.
A more demanding model is PMAC learning introduced by \cite{balcan2011learning}, where a learner has to output a hypothesis that multiplicatively approximates the target function. Specifically, a PMAC learner with approximation factor $\alpha$ and error $\varepsilon$ is an algorithm which outputs a hypothesis $h$ that satisfies $\Pr_{x \sim \mathcal{P}}[h(x) \le f(x) \le \alpha h(x)] \ge 1-\varepsilon$. We say that $h$ multiplicatively $(\alpha, \varepsilon)$-approximates $f$ over distribution $\mathcal{P}$ in this case.
While in general both models do not make assumptions on the distribution $\mathcal{P}$, our analysis will be focused on the fixed, uniform distribution over $\{0,1\}^n$. In practice, for a data utility learning task, one typically has control over the sampling distribution of the training samples and therefore can set the sampling distribution to be uniform.

\section{Learning Data Utility Functions}

\subsection{Characterizing Data Utility Functions}
\label{sec:characterize-duf}

A data utility function maps a set of data points to a real number indicating the utility of the data set. In the ML context, the utility of a dataset is typically measured by the performance ML model trained on the dataset, such as test accuracy.
It has been empirically observed in prior works~\citep{wang2021oneround} that many data utility functions can be efficiently learned with a relatively small amount of samples. One conjecture for the cause of such efficient learnability is that data utility functions are close to submodular functions (e.g., Figure \ref{fig:approach-submodularity}) for many common learning algorithms ~\citep{wang2021oneround}.

\begin{table}[]
\centering
\begin{tabular}{@{}cccccccc@{}}
\toprule
\multirow{2}{*}{$\beta$} & \textbf{Synthetic} & \textbf{USPS}  & \textbf{IRIS} & \textbf{IRIS} & \textbf{MNIST} & \textbf{MNIST} & \textbf{CIFAR-10} \\
& \textbf{+Logistic}     & \textbf{+SVM}   & \textbf{+Logistic} & \textbf{+SVM} & \textbf{+Logistic} & \textbf{+CNN}   & \textbf{+CNN} \\ \midrule
$0$                & 68.15\%                 & 66.15\%     & 94.1\%      & 92.5\%       & 50.95\%        & 50.5\%       & 50.8\% \\ \midrule
$0.98$             & 99\%                   & 98.7\%      & 99.85\%     & 99.4\%             & 66.4\%         & 66.85\%      & 79.6\% \\ \midrule
$0.99$             & 100\%         & 100\%           & 100\%           & 100\%           &  97.9\%                       & 95.8\%       & 99.6\% \\ \bottomrule
\end{tabular}
\caption{
We randomly draw two data subsets $S, T$ where $S \subseteq T$ and a data point $j \in N \setminus T$, and check how many sampled pairs satisfy the ``relaxed'' submodularity in (\ref{eq:submodularity-relax}).
We shows the percentage of sampled pairs satisfy the constraint in (\ref{eq:submodularity-relax}) given certain $\beta$ for the all datasets and models whose data utility functions are claimed to be ``approximate submodular'' in previous literature \cite{wang2021oneround}, with the exact same settings.
$\beta = 0$ corresponds to the case of the original definition of submodularity in (\ref{eq:submodularity}).
}
\label{tb:check-submodularity}
\end{table}

We first check how well submodularity describes the properties of common data utility functions. Specifically, we uniformly sample some subsets from training data and evaluate their corresponding utility. We then calculate the percentage of these utility samples satisfying the submodularity condition in (\ref{eq:submodularity}). The result is provided in the first row of Table \ref{tb:check-submodularity}. It shows that submodularity is an overly stringent condition to describe data utility functions because for many learning algorithms and datasets, the majority of utility samples do not satisfy it.
Therefore, we propose a refined condition for modeling common data utility functions.
\begin{definition}[$\beta$-relaxed submodularity]
We say that a set function $f$ satisfies $\beta$-relaxed submodularity if for every $S \subseteq T \subseteq N$, and every $j \in N \setminus T$,
\begin{equation}
f(T \cup \{ j \}) - f(T) \le f(S \cup \{j\})-f(S) + \beta^{n-(|T|-|S|)}
\label{eq:submodularity-relax}
\end{equation}
\end{definition}
In particular, when $\beta < 1$, the bias term $\beta^{n-(|T|-|S|)}$ attempts to model the phenomenon that when the datasets are small, the contributions of an additional data point to the datasets have larger variance. Hence, when two data sets get more different in sizes, the contributions to two sets might deviate more from the exact submodularity property.
When $\beta = 0$, this condition reduces to exact submodularity definition in (\ref{eq:submodularity}). Table \ref{tb:check-submodularity} shows that this relaxed condition better aligns with the property of data utility functions in practice.
The following theorem shows that any set functions with range $[0, 1]$ and satisfy the relaxed submodularity condition is self-bounding.

\begin{theorem}
Every function $f: \{0, 1\}^n \rightarrow [0, 1]$ satisfies
$$
f(T \cup \{ j \}) - f(T) \le f(S \cup \{j\})-f(S) + \beta^{n-(|T|-|S|)}
$$
for all $S \subseteq T$ and $j \in [n] \setminus T$, then
$f$ is $\left( 2, \frac{2\beta}{1-\beta} \right)$-self-bounding.
\label{thm:relaxed-submodular-sb}
\end{theorem}
Note that we can easily transform a data utility function to range $[0, 1]$ through normalization, as data utility functions typically have fixed ranges, e.g., classification accuracy in percentage always lies in 0 to 100. Hence, the learnability problem for data utility functions can be reduced to that for $(a, b)$-self bounding functions.

\subsection{Learning Self-bounding Functions}
\label{sec:theory}
Existing works on learning self-bounding functions either make extra assumptions of monotonicity~\cite{feldman2020tight} or focus exclusively on $(a, 0)$-self bounding functions~\cite{feldman2016optimal}. As shown in Figure~\ref{fig:approach-submodularity}, data utility functions in general are not monotone. Moreover, as shown in Table~\ref{tb:check-submodularity}, they can be better characterized by self-bounding functions with non-zero $\beta$. Hence, the learnability results in prior works cannot be directly applied to data utility functions. In this work, we extend these results to a more general case of $(a, b)$-self-bounding functions and relax the monotonicity constraint.

Our first result shows that under the PMAC framework, self-bounding functions can be learned with constant-factor errors. Formally, we show the following result.

\begin{theorem}
There exists an algorithm $\mathcal{A}$ that given the access to random and uniform examples of an $(a, b)$-self-bounding functions $f^*: \{0, 1\}^n \rightarrow [L, U]$ where $a \ge 1/3, L > 0$, with probability at least $1-\delta$, outputs a function $h$ which is a multiplicative $\left(4\theta \log(1/\varepsilon)/L, \varepsilon \right)$-approximation of $f^*$ over the uniform distribution, where $\theta =
21a-1 + \sqrt{\left( \frac{7a}{2} - \frac{1}{6} \right)^2 + 2b \log (1/\varepsilon)}$. Further, $\mathcal{A}$ runs in time $\mathcal{O}(1)$ and uses $(U-L)^2 \log(2/\delta) / (\theta^2 \log^2(1/\varepsilon))$ examples.
\label{thm:pmac-bound}
\end{theorem}
The intuition of the proof is inspired by \citep{balcan2011learning} as follows. Since the value of any self-bounding functions $f^*$ is tightly concentrated around its expectation under uniform distribution, the constant function equal the empirical mean gives a good approximation to $f^*$.
This result indicates that when we have access to random samples of a self-bounding function, it is possible to learn it up to constant multiplicative error with constant sample complexity and runtime. Note that in this result, there is a tradeoff between the approximation factor and error term, as constant function cannot approximate every self-bounding function arbitrarily well.

Our second learnability result is derived under a less demanding PAC learning framework.
We first extend the structural results in \cite{feldman2020tight} for $(a, 0)$-self-bounding functions to the more general setting of $(a, b)$-self-bounding functions. Specifically, we find that any $(a, b)$-self-bounding functions can be approximated by low-degree polynomials in $\ell_1$-norm.
\begin{theorem}
\label{thm:structure}
Let $f:\{0,1\}^{n} \rightarrow[0,1]$ be an $(a, b)$-self-bounding function and $\varepsilon>0$. For $d=O((a+b) / \varepsilon \cdot \log (1 / \varepsilon))$ there exists a set of indices I of size $2^{O(d)}$ and a polynomial $p$ of degree d over variables in $I$ such that $\|f-p\|_{1} \leq \varepsilon$.
\end{theorem}

Based on this structural result, we give a learnability result for $(a, b)$-self-bounding functions in a distribution-specific PAC learning model.
\begin{theorem}
\label{thm:structure-learn}
Let $\mathcal{C}_{(a, b)}$ be the class of all $(a, b)$-self-bounding functions from $\{0,1\}^{n}$ to $[0,1]$. Let $\rho = a+b$.
There exists an algorithm $\mathcal{A}$ that given $\epsilon>0$ and access to random uniform examples of any target function $f \in \mathcal{C}_{(a, b)}$, with probability at least $2/3$, outputs a hypothesis $h$, s.t. $\mathbb{E}[|f(x)-h(x)|] \le \varepsilon$.
Further, $\mathcal{A}$ runs in time $n^{\tilde{O}(\rho / \epsilon)}$ and uses $2^{\tilde{O}\left(\rho^{2} / \epsilon^{2}\right)} \log n$
examples.
\end{theorem}
The learning algorithm used to prove this theorem is based on polynomial $\ell_1$ regression over all monomials of degree $\Tilde{O}((a+b)/\varepsilon)$, inspired by \cite{feldman2016optimal} and \cite{feldman2020tight}.
Unlike Theorem \ref{thm:pmac-bound}, the approximation error can be arbitrarily small. Moreover, the sample complexity scale logarithmically with the input dimension $n$, which could still be quite efficient.

The practical implication of these learnability results is that, data utility functions could potentially be efficiently learned with limited amount of random samples of data utilities.

\section{Applying Data Utility Learning to Data Valuation}
\label{sec:dul-to-datavaluation}

As a concrete example to demonstrate the usefulness of data utility learning, we explore a novel application in data valuation.

Shapley value is a widely used data value notion nowadays \citep{ghorbani2019data, jia2019towards, jia2019efficient, jia2019scalability,  wang2020principled}. Recent work has also proposed to use the Least core\footnote{Least core may not be unique. In this paper, when we talk about the least core, we always refer to the least core vector that has the smallest $\ell_2$ norm, following the tie-breaking rule in the original literature \citep{yan2020ifyoulike}.} \citep{yan2020ifyoulike}, another famous solution concept in cooperative game theory, as a viable alternative to Shapley value. Both of the two data value notions have rigorous fairness guarantees, thereby making them particularly attractive in sensitive applications involving assigning monetary rewards or attributing responsibility based on data value. Given $n$ data points $D=\{z_1,\ldots,z_n\}$ and a data utility function $U$, the definition of Shapley value and Least core are respectively given as follows:
\begin{align}
   & s_{\text{shap}}(z_i) = \sum_{S\subseteq D\setminus\{z_i\}} \frac{1}{n{n-1 \choose |S|}}
\big[U(S\cup \{z_i\})-U(S)\big] \nonumber \\
&s_{\text{lc}} = \text{arg}\min_x e \quad \text{s.t.}~~\sum_{i=1}^n x_i = U(D), \sum_{z_i\in S}x_i +e \geq U(S), \forall S\subseteq D \nonumber
\end{align}
It can be seen from these definitions that the exact calculation of both the Shapley value and the least core requires evaluating the data utility function on every possible subset of the training data, which is $\mathcal{O}(2^n)$. Existing works on data valuation have largely focused on making the calculation more efficient. The major idea underlying the existing works is to evaluate the data utility only on some sampled subsets and then estimate the data value based on the utility samples. One key aspect ignored by all the prior works is that \emph{with the utility samples, we can potentially learn a model for the data utility function, which can be in turn used to predict the utility for subsets that are not sampled. }

This paper investigates the potential of using data utility learning to further improve the efficiency of various existing data value approximation heuristics. We first build an abstraction for data value approximation heuristics, which can incorporate all the existing \emph{unbiased} heuristics~\cite{maleki2015addressing, jia2019towards, yan2020ifyoulike} as well as some of the biased heuristics involving data utility sampling~\cite{ghorbani2019data}. These heuristics
consist of two components: a sampler $\mathscr{A}_{samp}$ and an estimator $\mathscr{A}_{est}$.
The heuristic sampler $\mathscr{A}_{samp}$ takes a dataset $D$ and sampling budget $m$ as the input, and outputs a sample set of data utilities $S_u = \{(S_i, v_i)\}_{i=1}^{m}$ where each $S_i \subseteq D, v_i = U(S_i)$.
The heuristic estimator $\mathscr{A}_{est}$ then takes the sampled utility set $S_u$ and compute the estimation of the corresponding solution concept (i.e., the Shapley value or the Least core). Algorithm \ref{alg:data-value-with-dul} summarizes our algorithm for accelerating data valuation with data utility learning. Our algorithm leverages the utility sample set $S_u$ already available in existing heuristics and use it for data utility learning (denoted as $S_{tr}$). Once we obtain the learned data utility model, we can use it to predict the utilities for more data subsets. At last, we use the combination of the true utility samples and predicted utility samples as input to $\mathscr{A}_{est}$. Since querying the utility model is very efficient,
the additional predicted utility samples can almost be acquired for free! In practice, \emph{evaluation budget} $m_{eval}$ could be set much greater than $m_{train}$ in Algorithm \ref{alg:data-value-with-dul}.

\begin{algorithm}
\SetAlgoLined
\SetKwInOut{Input}{input}
\SetKwInOut{Output}{output}
\Input{Data valuation heuristic $(\mathscr{A}_{samp}, \mathscr{A}_{est})$, Dataset $D$,
Training budget $m_{train}$, Evaluation budget $m_{eval}$, Training Algorithm for Data Utility Model $\mathscr{L}$.}
\Output{Value allocation vector $x \in \mathbb{R}^n$.}

Sample $m_{train}$ utilities of data subsets $S_{tr} = \{(S_i, v_i)\}_{i=1}^{m_{train}}$ with heuristic sampler $\mathscr{A}_{samp}$.

Train data utility model $f_u \leftarrow \mathscr{L}(S_{tr})$.

Sample $m_{eval}$ predicted utilities of data subsets $S_{pred} = \{(S_i, f_u(S_i))\}_{i=1}^{m_{eval}}$.

Compute the corresponding value allocation vector $x \leftarrow \mathscr{A}_{est} \left(S_{tr} \cup S_{pred} \right)$.

\Return{$x$}
\caption{Algorithm for Accelerating The Estimation of Data Valuation Solution Concepts}
\label{alg:data-value-with-dul}
\end{algorithm}

\paragraph{Comparison with Cooperative Game Abstraction. }
\cite{yan2020evaluating} propose a similar idea which approximates Shapley value through learning the characteristic functions in a cooperative game by using a parametric model called cooperative game abstraction (CGA). CGA is essentially a linear regression model where the variables are small subsets of players. The order of CGA refers to the largest size of player groups included as variables in the linear regression model. One advantage of CGA is that it can recover Shapley value directly from its parameters if the function of CGA perfectly matches the characteristic function. Hence, \cite{yan2020evaluating} propose to learn the characteristic function using CGA with certain amount of samples, and then compute Shapley value through the trained parameters of CGA.
For data valuation problem, the characteristic function is the data utility function. In this sense, \cite{yan2020evaluating}'s technique can be viewed as a special case of data utility learning where the data utility model is a linear regression. However, we argue that CGA may not be a suitable model for data utility learning with the two main reasons:
(i) CGA is only suitable for certain types of games where the interactions only exist for small groups of players, e.g., the team performance in basketball games.
On the contrary, interactions between large groups of data points still exist and might be strong. Thus, CGA is not a suitable model for data utility learning in nature, and we confirm this point in Section \ref{sec:eval}.
(ii) CGA has a poor scalability even for one with low order, e.g. for a third-order CGA with 2000 players, the number of total parameters is ${2000 \choose 2} + 2000 = 2,001,000$, while the number of total parameters of a $2000\times 512 \times 1$ fully-connected neural network is only only around half of it ($1,024,512$).
We also note that the parameters of CGA only has a close form for computing Shapley value, but not the Least core.

\section{Evaluation}
\label{sec:eval}
\subsection{Evaluation Settings}
\paragraph{Protocol.}
We evaluate the effectiveness of data utility learning for boosting the performance of the Shapley and Least core estimation heuristics. We first assess the performance of Shapley value and Least core estimators on small enough datasets to be able to directly calculate the true data value and then evaluate the estimation error of different heuristics.
For larger datasets, since it is impractical to compute the exact data value, we compare the performance of data value estimates on \emph{data removal} task, following~\cite{ghorbani2019data,jia2019efficient, jia2019scalability, wang2020principled, yan2020ifyoulike}.
Besides, we also evaluate the performance on data \emph{group} valuation, where the Shapley or least core values are assigned to a group of data rather than a single point. For all experiments, we train the data utility model using a three-layer MLP model.

\paragraph{Baselines.} For Shapley value estimation, we consider the two existing unbiased estimators as our baselines: \textbf{(1) Permutation Sampling (Perm)} \citep{maleki2015addressing}, which is a Monte Carlo algorithm;
\textbf{(2) Group Testing (GT)} \citep{jia2019towards}, which is an improved Monte Carlo algorithm based on group testing theory; and \textbf{(3) CGA}~\cite{yan2020evaluating}, which improve the efficiency of the Shapley value estimation by using a linear combination of the utilities on small subsets to estimate the utility function.
For the least core estimation, the baseline is the \textbf{Monte Carlo (MC)} approach \citep{yan2020ifyoulike}, which is the only known unbiased estimator.

We defer the implementation details of the data utility learning as well as baseline approaches to supplementary materials. For every experiment we conduct, we repeat each heuristic computation for 10 times to obtain the error bars.

\subsection{Results}
\label{sec:eval-results}
\textbf{Error Simulation. }
We first test the performance of different Shapley value or least core estimation heuristics on tiny datasets with less than 15 training data points. In this case, it becomes computationally feasible to compute the \emph{exact} Shapley and least core values and thus we can calculate the estimation error for different heuristics. We experiment on both synthetic and natural dataset.
To generate the synthetic dataset, we sample 10 data points from a 2-dimensional Gaussian distribution whose parameters are $0.1$ and $-0.1$, respectively. The labels are assigned based on the sign of the sum of the features. A logistic regression classifier trained on the 10 data points could achieve around 80\% test accuracy.
For the natural dataset, we randomly sample 15 data points from the famous Iris dataset \citep{pedregosa2011scikit}. A support-vector machine (SVM) classifier trained on the 15 data points achieve around 94\% test accuracy.
After training the utility model, we obtain the estimated data utilities for all the subsets not sampled before. We then estimate the Shapley and Least core values using the exact Shapley and Least core calculation formula, with the estimated data utilities.

\begin{comment}
\begin{wrapfigure}{R}{0.6\linewidth}
    \centering
    \includegraphics[width=0.6\textwidth]{images/error-sv.png}
    \caption{$\ell_\infty$ approximation error for Shapley
    values with the change of number of samples. DUL stands for data utility learning.}
    \label{fig:error-sv}
\end{wrapfigure}

\begin{wrapfigure}{R}{0.6\linewidth}
    \centering
    \includegraphics[width=0.6\textwidth]{images/error-lc.png}
    \caption{$\ell_\infty$ approximation error for
    Least core values with the change of number of samples. DUL stands for data utility learning.}
    \label{fig:error-lc}
\end{wrapfigure}
\end{comment}

\newcommand{0.6}{0.6}
\begin{wrapfigure}{R}{0.6\linewidth}
    \centering
    \includegraphics[width=0.6\textwidth]{images/error_22.png}
    \caption{$\ell_\infty$ approximation error for Shapley and Least core
    values with the change of number of samples. (a), (c) for synthetic dataset with 10 data points, (b), (d) for Iris dataset with 15 data points.
    DUL stands for data utility learning.}
    \label{fig:error-simulation}
\end{wrapfigure}

\begin{comment}
\begin{figure}
    \centering
    \includegraphics[width=\textwidth]{images/error_sv_lc.png}
    \caption{$\ell_\infty$ approximation error for Shapley and Least core
    values with the change of number of samples. (a), (c) for synthetic dataset with 10 data points, (b), (d) for Iris dataset with 15 data points.
    DUL stands for data utility learning.}
    \label{fig:error-simulation}
\end{figure}
\end{comment}

We show $\ell_\infty$ estimation errors in Figure \ref{fig:error-simulation}, and defer the results for $\ell_1$ and $\ell_2$ errors to supplementary materials.
As we can see, with relatively small amount of sampled utilities (e.g., 200-300 for synthetic dataset and 500 for Iris dataset), data utility learning can significantly reduce the estimation errors for both the Shapley and Least core values.
Utility prediction per se may introduce additional computational costs; yet, these costs are often negligible compared to model retraining. Moreover, CGA-based Shapley estimation performs poorly, because there are high-order data interactions in data utility functions which CGA cannot successfully capture.

\textbf{Data Removal. }
We evaluate the Shapley/Least core value estimations on larger datasets by comparing the performance on data removal task. Specifically, we remove the most (least) valuable portion of dataset and see how the utility of the remaining dataset changes. Intuitively, a better data value estimate can better identify the importance of data points. Hence, when the data with the highest (lowest) value estimates are removed, a better data value estimation method would lead to a faster (slower) performance drop.
Similar to the previous experiment, we experiment on both synthetic and real-world datasets.
For the synthetic data generation, we sample 200 data points from a 50-dimensional Gaussian distribution, where the 50-dimensional parameters are sampled uniformly from $[-1, 1]$, and each data point is labeled by the sign of the sum of the data point vector. The utility of the dataset is defined by the test accuracy of a logistic regression classifier trained on the dataset.
For real-world data experiment, we select 2000 data points from PubFig83 \citep{pinto2011scaling} dataset and the utility refers to the Top-5 accuracy of a convolutional neural network trained on it (for facial recognition).
Since in error simulation experiment we shown that CGA does not perform well in estimating Shapley values for data valuation, and since CGA is not scalable to larger datasets as discussed in Section \ref{sec:dul-to-datavaluation}, we do not compare it as a baseline for this experiment.

We experiment with different training budget $m_{train}$, and we set the evaluation budget $m_{eval}=10m_{train}$ as utility model evaluation is much faster than retraining the model.
We show the result of $m_{train}=500$ for synthetic data and $m_{train}=2500$ for PubFig83.
This is clearly a low-resource setting, as computing the exact Shapley or least core require $2^{200}$ times of training on synthetic data and $2^{2000}$ times of training on PubFig83 data.
As can be seen in Figure \ref{fig:removal-group} (a)-(b) and (d)-(e), the estimation heuristics equipped with data utility learning consistently performs better on identifying the most and least valuable data points.
This means that the Shapley and Least core values estimated with predicted utilities are at least more effective in predicting the most and least valuable (in a sense) data points in these settings.
As a side note, the Shapley value estimated by Permutation sampling is superior to the Least core estimated by Monte-Carlo algorithm, which does not agree with the experiment results in \cite{yan2020ifyoulike}. Therefore, an interesting future direction is to better understand the cases where Shapley value performs better than least core and vice versa.

\begin{figure}
    \centering
    \includegraphics[width=\textwidth]{images/removal-group.png}
    \caption{Curves of model test performance when the best/worst data points (groups) ranked according to Shapley (\textbf{SV}) or Least core (\textbf{LC}) estimations are removed.
    (a), (d) for Synthetic dataset, (b), (e) for PubFig83 dataset, (c), (f) for Adult data groups.
    The upper row ((a)-(c)) removes the best data points (groups).
    The steeper the drop, the better.
    The bottom row ((d)-(f)) removes the worst data points (groups).
    The sharper the rise, the better.
    DUL stands for data utility learning.
    }
    \label{fig:removal-group}
\end{figure}

\begin{comment}
\begin{figure}
    \centering
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/data_removal/synthetic_best_500.png}
        \label{fig:synthetic-removebest-500}
    \end{subfigure}
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/data_removal/synthetic_best_1000.png}
        \label{fig:synthetic-removebest-1000}
    \end{subfigure}
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/data_removal/synthetic_best_2000.png}
        \label{fig:synthetic-removebest-2000}
    \end{subfigure}
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/group_data/adult_best_500.png}
        \label{fig:adult-removebest-500}
    \end{subfigure}
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/group_data/adult_best_1000.png}
        \label{fig:adult-removebest-1000}
    \end{subfigure}
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/group_data/adult_best_2000.png}
        \label{fig:adult-removebest-2000}
    \end{subfigure}
    \caption{Curves of logistic regression test performance when the best and worst data points ranked according to the solution concepts are removed. In (a), (b), (c) the best data points are removed: the steeper the drop, the better.
    }
    \label{fig:dataremoval-synthetic}
\end{figure}
\end{comment}

\begin{comment}
\begin{figure}
    \centering
    \begin{subfigure}[b]{0.49\textwidth}
        \includegraphics[width=\textwidth]{images/uci_best_2000.png}
        \caption{Remove Best Data}
        \label{fig:uci-removebest-2000}
    \end{subfigure}
    \begin{subfigure}[b]{0.49\textwidth}
        \includegraphics[width=\textwidth]{images/uci_worst_2000.png}
        \caption{Remove Worst Data}
        \label{fig:uci-removeworst-2000}
    \end{subfigure}
    \caption{UCI}
    \label{fig:uci}
\end{figure}
\end{comment}

\textbf{Group Data Removal. }
We also experiment on estimating Shapley and Least core values for groups of data points. This is a potentially more realistic and useful setting since in practice, more than one data records will be collected from one party.
We divide Adult dataset \citep{adultdata} into 200 groups. The size of each group is varied. The proportion of data points allocated to each group is sampled from Dirichlet distribution where $\alpha$ has 30 in all dimensions. This design ensures that there are moderate amount of variations in group sizes. A utility sample in this setting refers to the performance of a logistic regression trained on the data points provided by a coalition of groups. This is a more challenging setting for data utility learning in nature, since the ``diminishing return'' property of data utility function may be violated due to the variation of group sizes.

Since Adult dataset is a highly unbalanced dataset, we use F1-score as the utility metric.
We show the results for $m_{train}=500, m_{eval}=5000$ here and defer the other settings to supplementary materials.
As we can see from Figure \ref{fig:removal-group} (c) and (f), the heuristics equipped with data utility learning is again favorable to both of the Shapley and Least core estimation.

\begin{comment}
\begin{figure}
    \centering
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/group_data/adult_best_500.png}
        \caption{Remove Best Data}
        \label{fig:adult-removebest-500}
    \end{subfigure}
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/group_data/adult_best_1000.png}
        \caption{Remove Best Data}
        \label{fig:adult-removebest-1000}
    \end{subfigure}
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/group_data/adult_best_2000.png}
        \caption{Remove Best Data}
        \label{fig:adult-removebest-2000}
    \end{subfigure}
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/group_data/adult_worst_500.png}
        \caption{Remove Worst Data}
        \label{fig:adult-removeworst-500}
    \end{subfigure}
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/group_data/adult_worst_1000.png}
        \caption{Remove Worst Data}
        \label{fig:adult-removeworst-1000}
    \end{subfigure}
    \begin{subfigure}[b]{0.6\textwidth}
        \includegraphics[width=\textwidth]{images/group_data/adult_worst_2000.png}
        \caption{Remove Worst Data}
        \label{fig:adult-removeworst-2000}
    \end{subfigure}
    \caption{Curves of logistic regression test performance when the best and worst data points ranked according to the solution concepts are removed. In (a), (b), (c) the best data points are removed: the steeper the drop, the better. In (d), (e), (f) the worst data points are removed: the sharper the rise, the better.}
    \label{fig:groupremoval-adult}
\end{figure}
\end{comment}

\section{Conclusion}
\label{sec:conclusion}
This paper presents the first learnability analysis for data utility functions. We propose a relaxed submodularity notion that can well describe the properties of data utility functions for popular ML models. We show that the relaxed submodular functions belong to self-bounding functions and then characterize the bounds on learning error under both PMAC and PAC learning models. Finally, we study the application of data utility learning in data valuation and propose a generic framework that can significantly improve the accuracy of all existing unbiased Shapley/least core estimation methods.

\textbf{Limitations \& Future Work. }
It is worth noting that many settings studied in our experiments in Section \ref{sec:eval} go much beyond the scope of our theoretical learnability results in Section \ref{sec:theory}. Specifically, the theoretical results are derived from simple learning algorithms such as $\ell_1$ regression and focused on uniform utility sample distribution. Meanwhile, our empirical results suggest that learning of data utility function could be data-efficient with neural network-based models, non-uniform utility sample distribution, and even functions not so close to being submodular.
Closing these gaps between theory and practices would be important future works.
With this paper, we march one step closer towards rigorous understanding of data utility learning and hope to inspire more research in this direction in ML community.

\newpage

\bibliographystyle{alpha}

redpajama_set_name: RedPajamaArXiv


For those of you not familiar with the work of cartoonist Chris Ware, one simply cannot heap enough praise on him as a creator.
His clean, design-inspired, almost clinical, architectural style of drawing merges perfectly (almost eerily) with his cleverly paced, if not somewhat disjointed, nonlinear storytelling style to create a totally unique and immersive reading experience.
Mr. Ware's books are more about wandering through a character's memories than simply reading a narrative (though the narrative inherent in and from these experiences is what makes them such masterful stories). A curious mixture of curiosity and pathos propels the reader tumbling through page after page to devour each and every subtle detail, searching out its meaning.
As a long-time fan of Mr. Ware's work, having all this finally collected and presented in one grand scope is something of a singular treat: Opening the box was like opening a little box of treasures, and I couldn't wait to dive in.
Unlike most comic creators, Mr. Ware insists on telling stories unapologetically grounded in the mundane (except when he's being deliberately surreal, those quirky stories are equally enjoyable). And though many of his contemporary cartoonists like Joe Matt, Adrian Tomine, or Seth also share a similar tone, Chris Ware is in a category all to himself in the depth and emotional resonance he is able to evoke from his characters.
Chris Ware is one of the true modern masters of the sequential art medium and an absolute artisan when it comes to showing the beauty of an ugly truth.
Building Stories collects over a decade's worth of Mr. Ware's work, as well as new and previously unpublished material, and revolves primarily around the residents of a three-story apartment building in Chicago.
It is comprised of 14 different books, pamphlets, mini-comics, newspaper-style fold outs, and more—all collected in a beautiful hardcover box that retails for a shockingly inexpensive $50.00 (a collection of this magnitude is worth three or even four times that).
Additionally, the various books can be read in any order, and the reader assembles the pieces of the characters' lives like some great puzzle, each layer weaving throughout the others. Indeed it's reminiscent of finding a box in the attic of things that belonged to someone you never really knew and piecing together their story.
Reading Building Stories feels like you are slowly assembling the pieces of an intricate and delicate clock whose pieces are all running off the tightly wound spring of angst. It's truly masterful storytelling that will be a unique experience for each reader and something that would be impossible in any other medium than print.
Book after book, Mr. Ware builds stories upon stories, some bizarrely different (wait until you get to the existential "children's book"), but all of them are intrinsically tied together with the common threads of both immediate proximity and vast isolation.
Periodically punctuated with vignettes of the brief yet defining moments that shape us, these serve as but brief interludes as more often we are delightfully subjected to the long metered pace of the everyday lives born out of those consequences.
Mr. Ware's characters all feel they live lives of longing, loneliness, and quiet desperation, each dwelling on regrets of missed opportunities and failed dreams, often suffering from a crippling sense of wasted time and wasted lives. But our perspective grows as we read further, and as we discover different facets to the same characters, the revelations become even more poignant and at times unsettlingly ironic—especially when we see how one person's failings can lead him to be so unintentionally cruel to another.
Any sentient reader can relate to the deep honesty we see in these character's private moments and in their private thoughts.
Each of us in our own way is desperately searching for our own sense of meaning, accomplishment, and self-worth, but anyone who has ever felt their creativity suppressed—or really anyone who has ever clung onto the good in their bad relationship (or the bad in their good relationship)—will see a lot of truth nestled in these almost magical pages.


After five months of controversy, the Supreme Court of PA made the decision to overturn a judge's previous decision to keep rapper Meek Mill in prison even when presented with evidence of a dirty cop.
The court made the decision to overturn Judge Genece Brinkley who was severely criticized for what some say was a grudge against Meek and keeping him in prison, despite prosecutors asking that his conviction be overturned because of a dirty cop.
The justices released Meek with no bail, pending a hearing in about 60 days to possibly dismiss the conviction. However, they did not dismiss Judge Brinkley from the case.
Mill was sentenced in November to two to four years for violating probation on a roughly decade-old gun and drug case.