A majority of people desires a stunning smile that radiates confidence. A smile not only gives you a unique look, but also impact greatly on your daily life. Thus, if your teeth are in bad shape that you feel shy to smile, then you certainly need something to revamp your confidence. With Invisalign, achieving that million-dollar smile is no longer an illusion.
Invisaligns are a series of custom-made aligners meant to straighten teeth. They are comfortable and gently shift teeth into place. With no metal brackets and wires to tighten, you just put in a new set of Invisalign aligners over your teeth every two weeks until you complete your treatment.
When you've got gaps or widely spaced teeth within your jaw and left untreated, it can lead to periodontal problems and increased risk of gum diseases.
Underbites occur when the lower teeth protrude beyond the front upper teeth, preventing the normal functioning of the molars. An overgrown upper or lower jaw or both are the contributing factors to this situation.
Patients who experience open bites cannot make contact between one set of teeth and the other opposite set. It is most often the result of a genetic disorder of the jaw structure or too much sucking of the thumb. Severe open bites can cause speech impairment.
Crowded teeth occur when one's teeth don't perfectly fit within the jaw structure. If left untreated, crowded teeth can develop into severely crooked teeth.
Crossbites occur when your lower and upper jaws are unevenly aligned. It typically causes your upper teeth to bite to the inside of your lower teeth.
The treatment process begins with you selecting an Invisalign-trained provider. This is critical because not all dentists or orthodontists specialise in working with Invisalign. During the consultation, the dentist will as well determine if you are a good case for Invisalign treatment.
Invisalign treatment typically lasts an average of one year for adults. The treatment plan will involve creating a 3-D image of the x-rays and the impressions of the teeth. The orthodontist will then map out the exact movement of the teeth at each stage.
The orthodontist will issue custom-made Invisalign aligners for you. They are made of smooth, comfortable, clear plastics that won't irritate your teeth, unlike metal braces. You simply need to wear them during the day and remove them only to eat, brush, or floss your teeth.
As you wear your aligners, regular checkups will be conducted after every six weeks to monitor the progress of the treatment. Invisalign aligners are worn 20-22 hours a day, and a new set replaced every two weeks. At every stage, your teeth will gently begin shifting into the right place. You'll experience the positive effects of healthier and better-functioning teeth and a rejuvenated smile.
Maintain proper oral hygiene by brushing and flossing your teeth regularly. Brush and rinse your aligners using lukewarm water or the Invisalign cleaning kit.
If you play sports that require wearing a mouth guard, remove your aligners before wearing the mouth guard.
Many are the people with conditions that demand Invisalign, while other on the same note, wished they had more info before they sorted for this treatment. Don't worry yourself, Trilliant Dentistry is your denture clinic in Milton that will not only enlighten you about Invisalign treatment option but also guarantees you a satisfactory outcome. Call us today to schedule a free consultation.
We offer a flexible timing schedule to accommodate our patients' needs. We also open on Saturdays.
We welcome our new patients with a gift. Please ask about the current new patients' offer when you visit our clinic for the first time.
For your convenience, we accept multiple forms of payments. We accept cash, major credit cards, debit cards, insurance etc.
You don't have to go through the pain of processing your insurance claims. We take this burden off your shoulder and process it on your behalf.
Once we receive your request, a staff member will contact you to confirm the appointment.
Trilliant Dentistry is a dental practice located in Milton, Ontario. We offer cosmetic, restorative, orthodontic. periodontic and many other general dental services. Our dentists and staff members strive to deliver high quality dental experience to our patients. You would feel this from your first to our practice.


Common Swift (Korscheltellus lupulina) (=Hepialus lupulinus) - Norfolk Moths - The macro and micro moths of Norfolk.
Most common of our swift moths. Female larger than the male, markings on males also vary greatly.
Grassland, gardens, roadsides, fens etc.
Do you have records of Common Swift?


text:
È considerato il padre della semeiotica percussoria delle malattie acute e croniche del torace.

Biografia

Giovinezza
Joseph Leopold Auenbrugger nacque a Graz, in Austria, da una famiglia benestante, il padre era proprietario di un'osteria. Fin da ragazzo, Auenbrugger mostrò uno spiccato orecchio musicale, tanto che il padre lo indusse a studiare il violino. Proprio grazie a questa sua dote naturale, lavorando nella bottega paterna, era divenuto molto bravo nel valutare il livello del vino nelle botti, percuotendone le pareti con piccoli colpi delle dita, infatti, era in grado di percepire anche le più sottili differenze di tonalità prodotte.

Gli studi universitari e la professione medica
Tuttavia il padre immaginava per il proprio figlio un futuro diverso da quello dell'oste, e così lo mandò all'Università di Vienna per studiare Medicina, presso la Scuola del famoso Gerhard Van Swieten. Auenbrugger provò così tanta ammirazione per il suo Maestro che, nel 1776, gli dedicò un lavoro intitolato Sugli impieghi della canfora nel trattamento delle manie.

Ottenne la laurea il 18 novembre del 1752, ma la sua esperienza in ospedale aveva avuto inizio già un anno prima. Dal 1751 al 1758, infatti, lavorò come assistente alla Spanische Krankenhaus, ospedale inizialmente fondato per curare persone di nazionalità spagnola, italiana e olandese, ma al tempo piena di soldati malati; e dal 1758 al 1762 ne fu primario. Lì Auenbrugger dedicò tutto se stesso allo studio delle malattie polmonari e mise a punto il nuovo metodo semeiotico di percussione, sfruttando ciò che aveva appreso da ragazzo nella bottega paterna.

La sua grande scoperta
Auenbrugger viene definito padre e maestro della percussione medica. In realtà anche Ippocrate, duemilacinquecento anni fa, utilizzava la percussione per rilevare qualche anomalia nel torace e nell'addome. La sua, però, era una tecnica approssimativa e priva di basi scientifiche; mentre la tecnica utilizzata dal medico viennese è basata su un'accurata e dettagliata sperimentazione.
Si tratta di una tecnica che permise ad Auenbrugger di percepire le differenze di risonanza tra le diverse parti del corpo. Egli percuoteva direttamente il petto con la punta delle sue dita, senza ricorrere all'altra mano come intermediario, come verrà fatto in seguito. Osservò per sette anni queste differenze di tonalità, in particolare nelle malattie dei polmoni e del cuore, confrontando e controllando i dati raccolti con quelli anatomopatologici (autoptici) rilevati nei pazienti deceduti e negli animali da esperimento.
Nel 1761, Auenbrugger organizzò tutti questi dati in un breve comunicazione in latino, dal titolo "Inventum novum".

Le critiche dei colleghi

Dissensi
I medici del tempo non compresero affatto l'importanza delle osservazioni di Auenbrugger e il vantaggio che loro stessi avrebbero potuto trarne. Anzi, la nuova scoperta suscitò moltissime critiche negative da parte dei colleghi. Rudolph Augustin Vogel disse che nell'Inventum Novum non ritrova nient'altro che la già tanto nota succussio Hippocrati. Perfino Gerhard van Swieten, maestro di Auenbrugger all'Università di Vienna, non apprezzò il suo lavoro. All'interno dell'ospedale vi era ormai un clima di grande tensione tanto che, nel 1762, il medico viennese dovette dimettersi dall'ospedale dedicandosi, così, esclusivamente alla professione privata.

Consensi
Ma, nonostante i dissensi, furono molti i giornali che parlarono in maniera positiva del nuovo metodo: il grande medico e poeta Albrecht von Haller, nel 1762, prestò molta attenzione a, come egli stesso lo definì, questo importante metodo in una recensione sul Göttingische Anzeigen von Gelehrten Sachen; Carl Friedrich Ludwig ne diede addirittura una definizione profetica, considerando la scoperta una torcia che porterà la luce nell'oscurità delle malattie del torace. Purtroppo queste furono le uniche grandi voci a favore di Auenbrugger nella seconda metà del settecento. Così il suo libro cadde ben presto nell'oblio.
Solo dopo cinquant'anni Jean-Nicolas Corvisart Des Marets, medico personale di Napoleone Bonaparte, riscoprì le poche pagine scritte da Auenbrugger, le tradusse dal latino al francese, ne verificò i dati e li commentò, realizzando così un testo quattro volte più copioso dell'originale. In questo modo nel 1808, un anno prima della morte del medico viennese, il metodo della percussione venne rivalutato, tanto che, non solo tutti i medici iniziarono ad utilizzarlo, ma iniziò anche ad essere insegnato nelle università.

La professione privata e gli ultimi anni
Al di là della sua grande scoperta, Auenbrugger era considerato un medico di gran successo, che però acquisì soprattutto dopo essersi dedicato esclusivamente alla professione privata. Tra i suoi pazienti figuravano sia personalità di spicco appartenenti alla nobiltà viennese, sia persone del popolo, peculiarità che contribuì a definirne la statura morale. Il 12 novembre del 1783, l'Imperatrice Maria Teresa elevò Auenbrugger al rango di Edler, cioè nobile. Così facendo il suo cognome poteva essere preceduto dall'ambito titolo von. Da allora il suo nome completo divenne Joseph Leopold Auenbrugger, Edler von Auenbrugg.
Si narra che il medico viennese aveva previsto la sua stessa morte: poco prima di mezzogiorno del 18 maggio, egli, osservando le sue condizioni e guardando l'orologio, dichiarò che sarebbe morto alle 14:00. Morì a Vienna il 18 maggio del 1809, all'età di 87 anni a causa di una polmonite, senza poter assistere alla fama mondiale che il suo nome avrebbe raggiunto poco tempo dopo.

Opere

Inventum Novum

Nel 1761 Auenbrugger riassunse tutti i dati raccolti in una breve comunicazione in latino: Inventum novum (Titolo completo: Inventum novum ex percussione thoracis humani ut signo abstrusos interni pectoris morbos detegendi). Un compendio di appena 1200 parole composto da 14 osservazioni, alcune di carattere generale, molte invece più specifiche.
Nell'introduzione afferma con molta modestia:

Nel suo lavoro Auenbrugger mise in evidenza, per la prima volta, come, nelle malattie del torace, il tono risonante, che normalmente si produce alla percussione, si modifica in sonus alterior (alto o timpanico), in sonus obscurior (suono indistinto) o in sonus carnis percussae (suono ottuso). Queste variazioni di tono consentono di differenziare con grande precisione le malattia del cuore e dei polmoni.
La dimostrazione del fatto che l'esperienza in bottega abbia aiutato molto Auenbrugger è data da alcuni versi della sua comunicazione qui riportati:

Il libro di Auenbrugger viene pubblicato in concomitanza con il De sedibus et causis morborum di Giovanni Battista Morgagni, dedicato anch'esso allo studio anatomopatologico delle malattie.

Opere minori
 "Experimentum nascens de remedio specifico sub signo specifico in mania virorum", 1776 (Sugli impieghi della canfora nel trattamento delle manie).
 "Von der stillen Wuth oder dem Triebe zum Selbstmorde als einer wirklichen Krankheit, mit Original-Beobachtungen und Anmerkungen." Dessau, 1783.
 "Heilart der epidemischen Ruhr." 1783.

Altre attività
Joseph Leopold von Auenbrugger, oltre a essere un grande medico, era appassionato di musica. Scrisse, nel 1781, per Antonio Salieri il libretto dei Singspiel in tre atti Der Rauchfangkehrer (Lo Spazzacamino), ottenendo anche un discreto successo. L'opera sarà rappresentata in prima al Burgtheater di Vienna e poi replicato a Francoforte, Berlino, Praga, Budapest, Riga.

Note

Bibliografia

Altri progetti

Collegamenti esterni

redpajama_set_name: RedPajamaWikipedia


Home › Why does the course not load?
Why does the course not load?
The main cause of these issues is the device's security protocols conflicting with the course. If you have an IT department we strongly recommend that they put the course into a 'whitelist', this will ensure that the course is unhindered by security.


At the VSTE conference this year, I attended a session where the presenter, Kristine Vester (@kavester) shared her ideas for Hybrid Breakouts. Basically, Hybrid Breakouts were a mix of digital and hands-on Breakout Edu games. Students solved all the clues online, and then once they knew the lock combinations, they used those combinations on a physical box with actual locks. It was appealing to me because it meant you could use less boxes (between groups you could re-lock the box), it allowed for more participation for individual players, and the set-up was a lot easier on me! I still LOVE hands-on Breakout Edu games and will continue to use them, but this idea provided an alternative, especially when there was limited time to make physical game pieces.
I tried it out with two fourth grade Virginia Studies classes. I built a Jamestown Digital Breakout using Breakout Edu's new platform. There were enough locks in the Digital version that every student on a team would be able to "drive" the computer for a lock.
At the end of the game, the reveal was the lock combinations for the physical box. There were enough locks on this one too that everyone in a group could open at least one lock.
After playing it a few times, I've adjusted the clues a bit. I think it's perfect for students in Grades 4 and 5, which was just what I needed for our Virginia SOLS. All teams were able to Breakout in under 45 minutes, and they had a blast playing!
So, just in case someone else would like to use it, included a link below. For the Hybrid version, you will also need a Breakout Box with a 4 digit lock, 3 digit lock, word lock, direction lock, and key lock (combinations are shown at the end of the game).
I went ahead and created a completely digital version of the game too, just in case someone without access to a box would like to play.
I don't want to post answers online, but if you need help figuring it out, feel free to contact me.


Upgrades from 4.2.1, 4.4.1, 5.0.0, and 5.2.0 to 5.4.1 are supported. Installations running previous versions of NNM 4.2.1 must upgrade to at least 4.2.1 prior to upgrading to NNM 5.5.1 Refer to the NNM 5.5.1 User Guide for details on upgrading to NNM 5.5.1.
NNM 5.5.1 is compatible with SecurityCenter 4.7.x and later.
NNM 5.5.1 is compatible with Industrial Security 1.1.1.
The NNM Web Server and NNM HTML5 User Interface are each automatically updated to version 2.1.1 via a plugin update.


The SCRIP Program is an easy system of using gift cards to pay for your purchases instead of cash. HCS families purchase gift cards for places they plan to shop or dine and use those cards to pay instead of cash or credit. The participating retailers then donate a percentage of what you spend to our school. This percentage is credited to you and is deducted from your tuition payments.
SCRIP orders are due at the school or online by 10:00 am on Wednesdays. Orders will be ready for pickup the following Wednesday morning.


And for one time free activation, you need to provide us picture to show us that CN900 MINI and The Key OBD PRO are on the same picture.
TOYOT Key OBD connect with cn900mini by Bluetooth, and operate on cn900mini screen.
Step 3: Cn900mini main interface display as pic4 (CN900 Mini will require to enter activation code), please take a photo of this and send to the salesmen which you contact, our salesmen will send this picture to our engineer, they will calculate the activation code; after the customer get the activation code and insert, it will register successfully.


Dina Asher-Smith could not continue her winning form from the European Championships, finishing second in the 200m at the Birmingham Grand Prix.The Briton - who won triple gold in Berlin last week - was beaten by Shaunae Miller-Uibo of the Bahamas."The race was such high calibre," Asher-Smith, 22, told BBC Sport. "It was like being in an Olympic final."
Elsewhere, Laura Muir won the women's 1,000m but missed out on Dame Kelly Holmes' 21-year-old British record.Second was enough for Asher-Smith to secure a 200m berth at the Diamond League Grand Final, where she will also compete in the 100m.The Birmingham Grand Prix is the penultimate leg of the Diamond League series, with the Grand Final set to be held over two nights in Zurich and Brussels on 30-31 August.
Asher-Smith came into this race as the only woman in the field to run under 22 seconds this year, having clocked 21.89 seconds in winning the European 200m title in Germany.She could not repeat that feat in Birmingham however, finishing in 22.31 as Commonwealth Games champion Miller-Uibo won in a new meeting record time of 22.15.Dutch world champion Dafne Schippers - the Olympic silver medallist from Rio 2016 - finished third."I am happy I was able to come out and perform in such a stacked field full of girls who,apart from me and Dafne, have been resting up," said Asher-Smith of England."It has been a hectic week since Berlin so I am just looking forward to relaxing at home for a week or so before the next race in Zurich."
New European 1500m champion Muir had been feeling under the weather since her Berlin brilliance but easily took gold again in Birmingham.Leading from the off despite a stumble, the 25-year-old Scot won in a time of two minutes 33.92 seconds - little more than a second off Holmes' record set in 1997.Fellow Briton Adelle Tracey placed third, with Katie Snowden and Jemma Reekie fourth and sixth respectively."I just tried to give it my best shot but when I saw I was off the pace I knew it became about just trying to win the race and I did that," Muir said."The Europeans was always the big target and to deliver was really good. I'm happy with the season."Elsewhere, Meghan Beesley finished third in the women's 400m hurdles with fellow Briton Eilidh Doyle fourth.In the women's long jump, Shara Proctor was the best of the four-strong British contingent, finishing third.
Britain's Reece Prescod and American Christian Coleman both finished the men's 100m in 9.94 seconds but the win went to Coleman in a photo finish.It still marked a personal best for 22-year-old Prescod, with Noah Lyles of the United States - the fastest man in the world this year - third.Jamaican Yohan Blake - the 2011 world champion and 2012 Olympic silver medallist - was fourth while Britain's recently crowned European champion Zharnel Hughes was fifth."I can't ask for any more than a personal best in Birmingham," Prescod told BBC Sport.Hughes, 23, added: "It was one of those days that wasn't for me unfortunately. It wasn't the best race I've had, but that happens."In the men's 400m, Briton's Matt Hudson-Smith - who won European gold in Berlin - finished second, just five hundredths of a second behind American winner Fred Kerley.
London 2012 Olympic champion Greg Rutherford finished eighth in the men's long jump in what was the 31-year-old Briton's penultimate competition before retiring.Former world, European and Commonwealth Games champion Rutherford withdrew after three jumps with his best jump recorded at 7.43m.South Africa's reigning world champion Luvo Manyonga took gold with a meeting record of 8.53m.Rutherford, who will compete for the last time at the Great North CityGames in Newcastle next month, told BBC Sport: "These events are all about me giving back. "I've done about two and a half hours of autographs already."


text:
Q: Adding static files to Nuget package that gets published to wwwroot on webproject publish event I'm trying to build a NuGet package with static files (js, CSS and images) that can be referenced as a package in a web project. I want these files to get published to the publish directory when publish even is triggered.
My .nuspec config is
<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
  <metadata minClientVersion="3.3.0">
    <id>Test.Assets.Develop</id>
    <version>1.0.0</version>
    <authors>Bev</authors>
    <owners>Bev</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Assets</description>
    <copyright>Copyright 2017</copyright>
    <contentFiles>
            <files include="contentFiles\any\any\assets\**" buildAction="Content" copyToOutput="true" flatten="false" />
    </contentFiles>
  </metadata>
  <files>
        <file src="build\assets\**" target="contentFiles\any\any\assets"/>
  </files>
</package>

My static files gets published to <outputdir>/bin folder instead of <outputdir>.

A: As per MS documentation contents should be copied to content folder within nuget package.
Refer here: https://learn.microsoft.com/en-us/nuget/create-packages/creating-a-package
content files can be placed at content folder
You could use something like this:
  <files>
       <file src="build\assets\**" target="content\" />
  </files>

redpajama_set_name: RedPajamaStackExchange


Floor tiles are a great investment in your home. They're attractive, virtually never wear out and they're easily cared for. If you looking for different look, then go for unique tile flooring. There are some unique tile flooring ideas. For instance, combining a unique tile such inlay tile with standard flooring is one of the best ways to create an appealing interior that will really stand out from the crowd.


adult bunk beds ikea loft beds bunk beds.
sleep number headboard sleep number bed headboard exclusive sleep number bed parts headboard pictures and awesome rails store sleep.
modern wood bench return bench hutch modern hutch modern vintage and locally crafted furniture located in.
wingback headboard king tufted headboard king tufted.
boy nursery wall decor baby boy bedroom sets boys bedroom wall decor boys bedroom wall decor baby boy wall decor.
costco pet beds orthopedic dog bed signature pet nest bed with pillow price home designer pro 4 orthopedic dog bed.
aqua duvet cover paisley block print duvet cover shams bedding set aqua green duvet cover.
100 bamboo sheets oatmeal bamboo sheet sets folded.
full size headboard and frame full platform bed frame with storage platform beds solid wood full size headboard reclaimed frame queen full size wood headboard and frame.
fall throw blanket crochet afghan colorful throw blanket fall and winter warm multiple photo blanket.


Good afternoon! I am thrilled you dropped by for coffee. I am brewing a fresh pot of hazelnut coffee for us. Don't you just love the aroma? I hope you find the flavor is to your liking. If not, I have other options.
It's a lovely afternoon with summery breezes and scents of flowers and budding trees. Perfect for chatting while sitting on the back porch, don't you agree?
I'm so glad you're here. It has been a while and I think of you often. Hey mom, do you remember?
Thank you Clare. She loved to dress up. Quite the lady.
Lovely tribute to your mother. Sorry you didn't have her here with you longer. You have sweet memories to keep her in your heart.
Lovely memories of your mother to cherish.
Oh your post tugged at my heartstrings. Maybe it hit a spot as I'm in a bit of a melancholy mood after turning 40 this year..
I would love to be 40 again. It seemed to be a great age for knowing oneself. We must enjoy every moment of our lives, even the melancholy moments as they come and go for a reason.
A lovely remembrance. Lucky you to have had such a wonderful relationship with your Mom.
I love this tribute to your mother. It was beautiful!
This is just wonderful, I felt so teary and so touched!
Oh this is such a beautiful tribute Eugenia. You mother must have been a delightful lady. I love this remembrance.
Thank you Jacqueline for your lovely comment.
You have inspired me to write about my youth, but think your youth and mine will be viewed very differently. Delightful piece.
Thank you. I would think we all have different pasts, which makes us so unique. I could write a novel about mine, but would rather just share little bits and pieces.
I marked the like but it didn't seem enough. Marvelous poem with memories of your mother. Love the in context rhyme.
Thank you very much. Your comment means a lot to me.
What a sweet remembrance of your mother . Surely a very special lady .
She was indeed very special. Thank you for the hugs.
This will be an inspiring piece for me to think positively about my own mother's presence when the time comes. Thanks.
Nice post. My mom passed away when I was 22, so this resonates with me.
Thank you for commenting. It's tough to lose your mom at such a young age.
I always enjoy your coffee!
Thank you and you're welcome to stop by anytime.


text:
Learning Center Facebook Marketing
4 Key Takeaways from Day 2 of Mark Zuckerberg's Congressional Hearing
In the second of two Congressional hearings this week, Mark Zuckerberg, Co-founder, Chairman, and CEO of Facebook, was questioned for a heavy five hours by members of the House, yesterday.
Once again, lawmakers took their four-minute windows to criticize, yet also praise, the company Zuckerberg created and its efforts to regulate the content & data shared on its platform.
Although, not as prominent in Wednesday's hearing, Cambridge Analytica was still brought up as a topic of discussion or an example in some of the House members questions.
House members instead presented other concerns surrounding privacy, data ownership, and GDPR and freedom of speech -- things all digital marketers and tech companies should keep in mind.
Lawmakers also made sure to grill Zuckerberg on answers they weren't satisfied with, explaining that if they couldn't get clarification, the general public couldn't either.
To sum up this talk, I've assembled 4 key takeaways from the questions, answers, and banter of day two.
If you missed the live stream, feel free to check it out here (or, read below for the abbreviated version)!
1. Facebook Position on Collecting Non-user Data
Zuckerberg came under continuous fire regarding how and why Facebook tracks data around the web, especially for those not on its platform.
The CEO explained that "in general we collect data on people who are not signed up for Facebook for security purposes."
This is done to track users with more malicious intents in mind, for example, identity theft or for "shadow profiles" on Facebook as Representative Ben Luján raised in his questioning.
Luján also explained that, with this in mind, it would be ideal if non-Facebook users could better understand what data other websites were harvesting on them, to which Zuckerberg replied, "we've got to fix that."
Ideally, Facebook would give these non-users the option for the platform not to follow them, but how to do so, without losing track of users who have malicious intent, is something the company needs to be aware of.
2. Free Speech and Political Content Regulation
Many lawmakers on both sides of the table raised points surrounding how Facebook flags certain abusive content, serving content that doesn't address all political views and preventing the spread of extremist content.
Zuckerberg explained, as he has before, that 99% of content that is deemed terrorist/extremist is flagged immediately by the AI before it has the chance to reach most audiences.
Outside of this, Facebook has a counter-terrorism team of 200 which specializes in 30 languages. This team looks for patterns of information certain groups/people put out and blocks it if it's classified as violent/shocking.
Conservatives also raised points as to whether or not the platform favored more liberal political agendas, and is unfairly banning their voice. Members of the house cited a few examples of instances where this happened, although, Facebook did previously explain these were mistakes.
Zuckerberg continues to emphasize that he will continue to address and improve the AI that scans Facebook's content to better serve content on both types of political views. But, he also addressed that it isn't completely fair to cite some outlier examples when the platform can never be perfect.
3. The Transparency Into Your Data for Companies Who Retarget You
Throughout the hearing, Zuckerberg had to clarify multiple times how ad retargeting works on the platform and what types of data companies actually have visibility into. To clarify, members of the House were concerned that once Facebook users visited other sites where the platform was continuously tracking you, those companies now had personal data they could sell to other data brokers.
During the hearing, Zuckerberg clarified that companies are only able to set parameters on their end to show ads to say, users who have liked a specific thing on a site/Facebook, visited their site, have followed specific Facebook pages, etc. These companies do not physically gain access to personal information on users individually by actions such as these.
(Of course, these places would have more personal data on you if you converted on their site, but this data would live somewhere else.)
4. Facebook to Implement GDPR
Congressman Green took the time to ask Zuckerberg about Europe's new General Data Protection Regulation (GDPR) and whether or not the company had plans to implement the regulations on the platform.
Zuckerberg responded by explaining "Yes, Congressman. We believe that everyone around the world deserves good privacy controls. We have had a lot of these controls in place for years. The GDPR requires us to do a few more things and we are going to extend that to the world."
There were 3 points Zuckerberg emphasized that explained how Facebook would implement the regulations:
Making sure people have control over how each piece of information is used on the platform: This includes what that data actually is, what company can see it, and whether or not they want to delete that data or restrict access.
Making sure people understand the tools that are available to regulate this data: This doesn't mean having a setting page that that's hidden somewhere, but putting tools in front of people so they can make their own decisions, ultimately building trust among Facebook users. This also helps improve the experience overall.
Enabling innovation among competitive technologies: As the world moves quickly to test out newer technologies, there is the need to allow more sensitive data to be shared. But, more regulations around this could slow down these innovations and risk the US losing to foreign competitors. These technologies would need to require special consent from the user so they know what extra data they are sharing, but in a way that doesn't overburden these technologies.
What data/stats you want to see Facebook release?
One of the biggest takeaways I personally noticed was that Zuckerberg doesn't have the answer to every question, and it's normal that he wouldn't. Congress urged him to keep open communication with them and continue to answer questions that he himself couldn't give answers to. I'm looking forward to what data they will release and how that affects the current users of Facebook and the trust they have in the company.
Is there something you want to see released? Or do you still feel there are unanswered questions? I'd love to hear what you're thinking in the comments below!
How Facebook's news feed algorithm works and prioritizes content
Marketers, Facebook Analytics will be no more on June 30, 2021
Facebook ads vs. boosted posts: Which is more effective in 2021?
Google Ads vs Facebook Ads: Which is better for your ad strategy?
How to maximize lead generation via Facebook posts and ads
Facebook for Creators offers tactical video advice leading into the holidays
Facebook Ads removing 28-Day attribution model, defaulting to 7-day
Facebook and Instagram Messenger merge; expands marketing potential
3 costly mistakes brands make with Black Friday and Cyber Monday Facebook and Instagram ads
Facebook ad 20% text limit is now a thing of the past
Facebook launches e-learning program for community managers
Facebook Shop to support online retailers, e-commerce during COVID-19
Facebook is removing 1,000+ ad targeting options because of low usage
How to turn strangers into customers (and friends) with Facebook Ads [AdvertiseMint CEO Brian Meert Interview]
Boutique clothing brand sees 99% revenue increase with new paid ads strategy
Google: Black Friday and Cyber Monday will be different in 2020 for e-commerce
Apple's privacy update: What does it mean for your mobile ad strategy?
Data privacy update: How the CCPA affects you and your paid advertising
The North Face kicks off Facebook ad boycott
Facebook and Instagram will allow users to 'turn off' political ads ahead of 2020 election
The most important Facebook and Instagram ad benchmarks you need to know [Infographic]
Facebook Messenger will soon allow you to toggle between business and personal accounts (finally!)
Facebook confirms it's testing email marketing for business pages
8 best webinar software to go live on Facebook (free and paid)
What Facebook Ads problems do digital marketers need to watch out for?

redpajama_set_name: RedPajamaCommonCrawl


text:
Skype to block local voice calls originating from India starting November 10.
Yes, it's true and officially confirmed.
From the support page of Skype: https://support.skype.com/en/faq/FA34487/why-can-i-no-longer-make-calls-to-india-from-india
A Times of India article on the same: http://timesofindia.indiatimes.com/tech/tech-news/Skype-to-block-local-voice-calls-from-India-starting-November-10/articleshow/44479747.cms
According to Indian law, although it's legal to use VoIP, it's illegal to have VoIP gateways (server) in India. Read: The TRAI PDF whitepaper. The irony is that, the Government is forcing VoIP service providers to place VoIP gateways inside India while it's illegal to do so! Read: News article from Times of India – "Government wants Skype to set up servers in India".
Get my point here? The Government is forcing VoIP service providers to place gateways inside in India, while not amending the earlier policy which made it illegal. So in anyways, VoIPs can't offer local calls 'legally'. The contradictory polices of both TRAI and Government of India made VoIP service providers difficult to operate in India. So naturally, Skype had to stop the services.
What do you think of this move? Comment your opinion.
Further reading: http://en.wikipedia.org/wiki/Voice_over_IP
IndiaSkypeTRAIVoIP


The "Fruit Fly Trap" costs $7.99 and claims to catch fruit flies.
It has a baited center tube you fill with water, and also sticky trap pads you put on either side of the tube.
We figured by putting a bowl of fruit a couple of feet away from the trap, we wouldn't have any trouble catching flies.
Not a skinny bug/insect/critter of any kind met their doom in the Fruit Fly Trap.
If you have purchased the Fruit Fly Trap and had some success with it, let me know. I will happily update the story.


Don't miss this one, this home is updated and ready for you! New roof and windows, All new kitchen with granite counter tops, all new fixtures, updated bathrooms, fresh paint and more! Four bedrooms, Master bedroom has full bath and His & Her closets, Formal living room, formal dining room, Family room with fireplace, kitchen with breakfast area, covered front porch, rear patio in private backyard. super clean 1 car garage and more! Call today to schedule your showing.


text:
package org.jclouds.chef;

import java.io.Closeable;
import java.io.InputStream;
import java.net.URI;
import java.util.List;
import java.util.Set;

import javax.inject.Named;
import javax.ws.rs.Consumes;
import javax.ws.rs.DELETE;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

import org.jclouds.Constants;
import org.jclouds.Fallbacks.EmptySetOnNotFoundOr404;
import org.jclouds.Fallbacks.NullOnNotFoundOr404;
import org.jclouds.Fallbacks.VoidOnNotFoundOr404;
import org.jclouds.chef.binders.BindChecksumsToJsonPayload;
import org.jclouds.chef.binders.BindCreateClientOptionsToJsonPayload;
import org.jclouds.chef.binders.BindGenerateKeyForClientToJsonPayload;
import org.jclouds.chef.binders.DatabagItemId;
import org.jclouds.chef.binders.EnvironmentName;
import org.jclouds.chef.binders.NodeName;
import org.jclouds.chef.binders.RoleName;
import org.jclouds.chef.domain.Client;
import org.jclouds.chef.domain.CookbookDefinition;
import org.jclouds.chef.domain.CookbookVersion;
import org.jclouds.chef.domain.DatabagItem;
import org.jclouds.chef.domain.Environment;
import org.jclouds.chef.domain.Node;
import org.jclouds.chef.domain.Resource;
import org.jclouds.chef.domain.Role;
import org.jclouds.chef.domain.Sandbox;
import org.jclouds.chef.domain.SearchResult;
import org.jclouds.chef.domain.UploadSandbox;
import org.jclouds.chef.filters.SignedHeaderAuth;
import org.jclouds.chef.functions.ParseCookbookDefinitionCheckingChefVersion;
import org.jclouds.chef.functions.ParseCookbookDefinitionFromJsonv10;
import org.jclouds.chef.functions.ParseCookbookDefinitionListFromJsonv10;
import org.jclouds.chef.functions.ParseCookbookVersionsCheckingChefVersion;
import org.jclouds.chef.functions.ParseKeySetFromJson;
import org.jclouds.chef.functions.ParseSearchClientsFromJson;
import org.jclouds.chef.functions.ParseSearchDatabagFromJson;
import org.jclouds.chef.functions.ParseSearchEnvironmentsFromJson;
import org.jclouds.chef.functions.ParseSearchNodesFromJson;
import org.jclouds.chef.functions.ParseSearchRolesFromJson;
import org.jclouds.chef.functions.UriForResource;
import org.jclouds.chef.options.CreateClientOptions;
import org.jclouds.chef.options.SearchOptions;
import org.jclouds.io.Payload;
import org.jclouds.rest.annotations.BinderParam;
import org.jclouds.rest.annotations.EndpointParam;
import org.jclouds.rest.annotations.Fallback;
import org.jclouds.rest.annotations.Headers;
import org.jclouds.rest.annotations.MapBinder;
import org.jclouds.rest.annotations.ParamParser;
import org.jclouds.rest.annotations.PayloadParam;
import org.jclouds.rest.annotations.RequestFilters;
import org.jclouds.rest.annotations.ResponseParser;
import org.jclouds.rest.annotations.SelectJson;
import org.jclouds.rest.annotations.SinceApiVersion;
import org.jclouds.rest.annotations.SkipEncoding;
import org.jclouds.rest.annotations.WrapWith;
import org.jclouds.rest.binders.BindToJsonPayload;

import com.google.inject.Provides;

/**
 * Provides synchronous access to Chef.
 */
@RequestFilters(SignedHeaderAuth.class)
@Headers(keys = "X-Chef-Version", values = "{" + Constants.PROPERTY_API_VERSION + "}")
@Consumes(MediaType.APPLICATION_JSON)
public interface ChefApi extends Closeable {

   /**
    * Provides access to high level Chef features.
    */
   @Provides
   ChefService chefService();

   // Clients

   /**
    * Lists the names of the existing clients.
    *
    * @return The names of the existing clients.
    */
   @Named("client:list")
   @GET
   @Path("/clients")
   @ResponseParser(ParseKeySetFromJson.class)
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<String> listClients();

   /**
    * Gets the details of existing client.
    *
    * @param clientname The name of the client to get.
    * @return The details of the given client.
    */
   @Named("client:get")
   @GET
   @Path("/clients/{clientname}")
   @Fallback(NullOnNotFoundOr404.class)
   Client getClient(@PathParam("clientname") String clientName);

   /**
    * Creates a new client.
    *
    * @param clientname The name of the new client
    * @return The client with the generated private key. This key should be
    *         stored so client can be properly authenticated .
    */
   @Named("client:create")
   @POST
   @Path("/clients")
   @MapBinder(BindToJsonPayload.class)
   Client createClient(@PayloadParam("name") String clientName);

   /**
    * Creates a new client with custom options.
    *
    * @param clientname The name of the new client
    * @param options The options to customize the client creation.
    * @return The client with the generated private key. This key should be
    *         stored so client can be properly authenticated .
    */
   @Named("client:create")
   @POST
   @Path("/clients")
   @MapBinder(BindCreateClientOptionsToJsonPayload.class)
   Client createClient(@PayloadParam("name") String clientName, CreateClientOptions options);

   /**
    * Generates a new key-pair for this client, and return the new private key in
    * the response body.
    *
    * @param clientname The name of the client.
    * @return The details of the client with the new private key.
    */
   @Named("client:generatekey")
   @PUT
   @Path("/clients/{clientname}")
   Client generateKeyForClient(
         @PathParam("clientname") @BinderParam(BindGenerateKeyForClientToJsonPayload.class) String clientName);

   /**
    * Deletes the given client.
    *
    * @param clientname The name of the client to delete.
    * @return The deleted client.
    */
   @Named("client:delete")
   @DELETE
   @Path("/clients/{clientname}")
   @Fallback(NullOnNotFoundOr404.class)
   Client deleteClient(@PathParam("clientname") String clientName);

   // Cookbooks

   /**
    * Lists the names of the existing cookbooks.
    *
    * @return The names of the exsisting cookbooks.
    */
   @Named("cookbook:list")
   @GET
   @Path("/cookbooks")
   @ResponseParser(ParseCookbookDefinitionCheckingChefVersion.class)
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<String> listCookbooks();

   /**
    * Lists the cookbooks that are available in the given environment.
    *
    * @param environmentname The name of the environment to get the cookbooks
    *        from.
    * @return The definitions of the cookbooks (name, URL and versions) available in
    *         the given environment.
    */
   @SinceApiVersion("0.10.0")
   @Named("cookbook:list")
   @GET
   @ResponseParser(ParseCookbookDefinitionListFromJsonv10.class)
   @Path("/environments/{environmentname}/cookbooks")
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<CookbookDefinition> listCookbooksInEnvironment(@PathParam("environmentname") String environmentName);

   /**
    * Lists the cookbooks that are available in the given environment, limiting
    * the number of versions returned for each cookbook.
    *
    * @param environmentname The name of the environment to get the cookbooks
    *        from.
    * @param numversions The number of cookbook versions to include in the
    *        response, where n is the number of cookbook versions.
    * @return The definitions of the cookbooks (name, URL and versions) available in
    *         the given environment.
    */
   @SinceApiVersion("0.10.0")
   @Named("cookbook:list")
   @GET
   @ResponseParser(ParseCookbookDefinitionListFromJsonv10.class)
   @Path("/environments/{environmentname}/cookbooks?num_versions={numversions}")
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<CookbookDefinition> listCookbooksInEnvironment(@PathParam("environmentname") String environmentName,
         @PathParam("numversions") String numVersions);

   /**
    * Lists the available versions of the given cookbook.
    *
    * @param cookbookName The name of the cookbook.
    * @return The available versions of the given cookbook.
    */
   @Named("cookbook:versions")
   @GET
   @Path("/cookbooks/{cookbookname}")
   @ResponseParser(ParseCookbookVersionsCheckingChefVersion.class)
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<String> listVersionsOfCookbook(@PathParam("cookbookname") String cookbookName);

   /**
    * Gets the details of the given cookbook, with the links to each resource
    * such as recipe files, attributes, etc.
    *
    * @param cookbookName The name of the cookbook.
    * @param version The version of the cookbook to get.
    * @return The details of the given cookbook.
    */
   @Named("cookbook:get")
   @GET
   @Path("/cookbooks/{cookbookname}/{version}")
   @Fallback(NullOnNotFoundOr404.class)
   CookbookVersion getCookbook(@PathParam("cookbookname") String cookbookName, @PathParam("version") String version);

   /**
    * Gets the definition of the cookbook in the given environment.
    *
    * @param environmentname The name of the environment.
    * @param cookbookname The name of the cookbook.
    * @return The definition of the cookbook (URL and versions) of the cookbook
    *         in the given environment.
    */
   @SinceApiVersion("0.10.0")
   @Named("environment:cookbook")
   @GET
   @ResponseParser(ParseCookbookDefinitionFromJsonv10.class)
   @Path("/environments/{environmentname}/cookbooks/{cookbookname}")
   CookbookDefinition getCookbookInEnvironment(@PathParam("environmentname") String environmentName,
         @PathParam("cookbookname") String cookbookName);

   /**
    * Gets the definition of the cookbook in the given environment.
    *
    * @param environmentname The name of the environment.
    * @param cookbookname The name of the cookbook.
    * @param numversions The number of cookbook versions to include in the
    *        response, where n is the number of cookbook versions.
    * @return The definition of the cookbook (URL and versions) of the cookbook
    *         in the given environment.
    */
   @SinceApiVersion("0.10.0")
   @Named("environment:cookbook")
   @GET
   @ResponseParser(ParseCookbookDefinitionFromJsonv10.class)
   @Path("/environments/{environmentname}/cookbooks/{cookbookname}?num_versions={numversions}")
   CookbookDefinition getCookbookInEnvironment(@PathParam("environmentname") String environmentName,
         @PathParam("cookbookname") String cookbookName, @PathParam("numversions") String numVersions);

   /**
    * Lists the names of the recipes in the given environment.
    *
    * @param environmentname The name of the environment.
    * @return The names of the recipes in the given environment.
    */
   @SinceApiVersion("0.10.0")
   @Named("environment:recipelist")
   @GET
   @Path("/environments/{environmentname}/recipes")
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<String> listRecipesInEnvironment(@PathParam("environmentname") String environmentName);

   /**
    * Creates or updates the given cookbook.
    *
    * @param cookbookName The name of the cookbook to create or update.
    * @param version The version of the cookbook to create or update.
    * @param cookbook The contents of the cookbook to create or update.
    * @return The details of the created or updated cookbook.
    */
   @Named("cookbook:update")
   @PUT
   @Path("/cookbooks/{cookbookname}/{version}")
   CookbookVersion updateCookbook(@PathParam("cookbookname") String cookbookName, @PathParam("version") String version,
         @BinderParam(BindToJsonPayload.class) CookbookVersion cookbook);

   /**
    * Deletes the given cookbook.
    *
    * @param cookbookName The name of the cookbook to delete.
    * @param version The version of the cookbook to delete.
    * @return The details of the deleted cookbook.
    */
   @Named("cookbook:delete")
   @DELETE
   @Path("/cookbooks/{cookbookname}/{version}")
   @Fallback(NullOnNotFoundOr404.class)
   CookbookVersion deleteCookbook(@PathParam("cookbookname") String cookbookName, @PathParam("version") String version);

   // Data bags

   /**
    * Lists the names of the existing data bags.
    *
    * @return The names of the existing data bags.
    */
   @Named("databag:list")
   @GET
   @Path("/data")
   @ResponseParser(ParseKeySetFromJson.class)
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<String> listDatabags();

   /**
    * Creates a new data bag.
    *
    * @param databagName The name for the new data bag.
    */
   @Named("databag:create")
   @POST
   @Path("/data")
   void createDatabag(@WrapWith("name") String databagName);

   /**
    * Deletes a data bag, including its items.
    *
    * @param databagName The name of the data bag to delete.
    */
   @Named("databag:delete")
   @DELETE
   @Path("/data/{name}")
   @Fallback(VoidOnNotFoundOr404.class)
   void deleteDatabag(@PathParam("name") String databagName);

   /**
    * Lists the names of the items in a data bag.
    *
    * @param databagName The name of the data bag.
    * @return The names of the items in the given data bag.
    */
   @Named("databag:listitems")
   @GET
   @Path("/data/{name}")
   @ResponseParser(ParseKeySetFromJson.class)
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<String> listDatabagItems(@PathParam("name") String databagName);

   /**
    * Gets an item in a data bag.
    *
    * @param databagName The name of the data bag.
    * @param databagItemId The identifier of the item to get.
    * @return The details of the item in the given data bag.
    */
   @Named("databag:getitem")
   @GET
   @Path("/data/{databagName}/{databagItemId}")
   @Fallback(NullOnNotFoundOr404.class)
   DatabagItem getDatabagItem(@PathParam("databagName") String databagName,
         @PathParam("databagItemId") String databagItemId);

   /**
    * Adds an item in a data bag.
    *
    * @param databagName The name of the data bag.
    * @param The item to add to the data bag.
    * @param The item just added to the data bag.
    */
   @Named("databag:createitem")
   @POST
   @Path("/data/{databagName}")
   DatabagItem createDatabagItem(@PathParam("databagName") String databagName,
         @BinderParam(BindToJsonPayload.class) DatabagItem databagItem);

   /**
    * Updates an item in a data bag.
    *
    * @param databagName The name of the data bag.
    * @param item The new contents for the item in the data bag.
    * @return The details for the updated item in the data bag.
    */
   @Named("databag:updateitem")
   @PUT
   @Path("/data/{databagName}/{databagItemId}")
   DatabagItem updateDatabagItem(
         @PathParam("databagName") String databagName,
         @PathParam("databagItemId") @ParamParser(DatabagItemId.class) @BinderParam(BindToJsonPayload.class) DatabagItem item);

   /**
    * Deletes an item from a data bag.
    *
    * @param databagName The name of the data bag.
    * @param databagItemId The identifier of the item to delete.
    * @return The item deleted from the data bag.
    */
   @Named("databag:deleteitem")
   @DELETE
   @Path("/data/{databagName}/{databagItemId}")
   @Fallback(NullOnNotFoundOr404.class)
   @SelectJson("raw_data")
   DatabagItem deleteDatabagItem(@PathParam("databagName") String databagName,
         @PathParam("databagItemId") String databagItemId);

   // Environments

   /**
    * Lists the names of the existing environments.
    *
    * @return The names of the existing environments.
    */
   @SinceApiVersion("0.10.0")
   @Named("environment:list")
   @GET
   @Path("/environments")
   @ResponseParser(ParseKeySetFromJson.class)
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<String> listEnvironments();

   /**
    * Gets the details of an existing environment.
    *
    * @param environmentname The name of the environment to get.
    * @return The details of the given environment.
    */
   @SinceApiVersion("0.10.0")
   @Named("environment:get")
   @GET
   @Path("/environments/{environmentname}")
   @Fallback(NullOnNotFoundOr404.class)
   Environment getEnvironment(@PathParam("environmentname") String environmentName);

   /**
    * Creates a new environment.
    *
    * @param environment The environment to create.
    */
   @SinceApiVersion("0.10.0")
   @Named("environment:create")
   @POST
   @Path("/environments")
   void createEnvironment(@BinderParam(BindToJsonPayload.class) Environment environment);

   /**
    * Updates the given environment.
    *
    * @param environment The new details for the environment.
    * @return The details of the updated environment.
    */
   @SinceApiVersion("0.10.0")
   @Named("environment:update")
   @PUT
   @Path("/environments/{environmentname}")
   Environment updateEnvironment(
         @PathParam("environmentname") @ParamParser(EnvironmentName.class) @BinderParam(BindToJsonPayload.class) Environment environment);

   /**
    * Deletes the given environment.
    *
    * @param environmentname The name of the environment to delete.
    * @return The details of the deleted environment.
    */
   @SinceApiVersion("0.10.0")
   @Named("environment:delete")
   @DELETE
   @Path("/environments/{environmentname}")
   @Fallback(NullOnNotFoundOr404.class)
   Environment deleteEnvironment(@PathParam("environmentname") String environmentName);

   // Nodes

   /**
    * Lists the names of the existing nodes.
    *
    * @return The names of the existing nodes.
    */
   @Named("node:list")
   @GET
   @Path("/nodes")
   @ResponseParser(ParseKeySetFromJson.class)
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<String> listNodes();

   /**
    * Lists the names of the nodes in the given environment.
    *
    * @param environmentname The name of the environment.
    * @return The names of the existing nodes in the given environment.
    */
   @SinceApiVersion("0.10.0")
   @Named("environment:nodelist")
   @GET
   @Path("/environments/{environmentname}/nodes")
   @ResponseParser(ParseKeySetFromJson.class)
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<String> listNodesInEnvironment(@PathParam("environmentname") String environmentName);

   /**
    * Gets the details of the given node.
    *
    * @param nodename The name of the node to get.
    * @return The details of the given node.
    */
   @Named("node:get")
   @GET
   @Path("/nodes/{nodename}")
   @Fallback(NullOnNotFoundOr404.class)
   Node getNode(@PathParam("nodename") String nodeName);

   /**
    * Creates a new node.
    *
    * @param node The details of the node to create.
    */
   @Named("node:create")
   @POST
   @Path("/nodes")
   void createNode(@BinderParam(BindToJsonPayload.class) Node node);

   /**
    * Updates an existing node.
    *
    * @param node The new details for the node.
    * @return The details of the updated node.
    */
   @Named("node:update")
   @PUT
   @Path("/nodes/{nodename}")
   Node updateNode(@PathParam("nodename") @ParamParser(NodeName.class) @BinderParam(BindToJsonPayload.class) Node node);

   /**
    * Deletes the given node.
    *
    * @param nodename The name of the node to delete.
    * @return The details of the deleted node.
    */
   @Named("node:delete")
   @DELETE
   @Path("/nodes/{nodename}")
   @Fallback(NullOnNotFoundOr404.class)
   Node deleteNode(@PathParam("nodename") String nodeName);

   // Roles

   /**
    * Lists the names of the existing roles.
    *
    * @return The names of the existing roles.
    */
   @Named("role:list")
   @GET
   @Path("/roles")
   @ResponseParser(ParseKeySetFromJson.class)
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<String> listRoles();

   /**
    * Gets the details of the given role.
    *
    * @param rolename The name of the role to get.
    * @return The details of the given role.
    */
   @Named("role:get")
   @GET
   @Path("/roles/{rolename}")
   @Fallback(NullOnNotFoundOr404.class)
   Role getRole(@PathParam("rolename") String roleName);

   /**
    * Creates a new role.
    *
    * @param role The details for the new role.
    */
   @Named("role:create")
   @POST
   @Path("/roles")
   void createRole(@BinderParam(BindToJsonPayload.class) Role role);

   /**
    * Updates the given role.
    *
    * @param role The new details for the role.
    * @return The details of the updated role.
    */
   @Named("role:update")
   @PUT
   @Path("/roles/{rolename}")
   Role updateRole(@PathParam("rolename") @ParamParser(RoleName.class) @BinderParam(BindToJsonPayload.class) Role role);

   /**
    * Deletes the given role.
    *
    * @param rolename The name of the role to delete.
    * @return The details of the deleted role.
    */
   @Named("role:delete")
   @DELETE
   @Path("/roles/{rolename}")
   @Fallback(NullOnNotFoundOr404.class)
   Role deleteRole(@PathParam("rolename") String roleName);

   // Sandboxes

   /**
    * Creates a new sandbox.
    * <p>
    * It accepts a list of checksums as input and returns the URLs against which
    * to PUT files that need to be uploaded.
    *
    * @param md5s The raw md5 sums. Uses {@code Bytes.asList()} and
    *        {@code Bytes.toByteArray()} as necessary
    * @return The upload sandbox with the URLs against which to PUT files that
    *         need to be uploaded.
    */
   @Named("sandbox:upload")
   @POST
   @Path("/sandboxes")
   UploadSandbox createUploadSandboxForChecksums(@BinderParam(BindChecksumsToJsonPayload.class) Set<List<Byte>> md5s);

   /**
    * Uploads the given content to the sandbox at the given URI.
    * <p>
    * The URI must be obtained, after uploading a sandbox, from the
    * {@link UploadSandbox#getUri()}.
    *
    * @param location The URI where the upload must be performed.
    * @param content The contents to upload.
    */
   @Named("content:upload")
   @PUT
   @Produces("application/x-binary")
   void uploadContent(@EndpointParam URI location, Payload content);

   /**
    * Gets the contents of the given resource.
    *
    * @param resource The resource to get.
    * @return An input stream for the content of the requested resource.
    */
   @Named("content:get")
   @GET
   @Fallback(NullOnNotFoundOr404.class)
   @SkipEncoding({ '+', ' ', '/', '=', ':', ';' })
   InputStream getResourceContents(@EndpointParam(parser = UriForResource.class) Resource resource);

   /**
    * Confirms if the sandbox is completed or not.
    * <p>
    * This method should be used after uploading contents to the sandbox.
    *
    * @param id The id of the sandbox to commit.
    * @param isCompleted Flag to set if the sandbox is completed or not.
    * @return The details of the sandbox.
    */
   @Named("sandbox:commit")
   @PUT
   @Path("/sandboxes/{id}")
   Sandbox commitSandbox(@PathParam("id") String id, @WrapWith("is_completed") boolean isCompleted);

   // Search

   /**
    * Lists the names of the available search indexes.
    * <p>
    * By default, the "role", "node" and "api" indexes will always be available.
    * <p>
    * Note that the search indexes may lag behind the most current data by at
    * least 10 seconds at any given time - so if you need to write data and
    * immediately query it, you likely need to produce an artificial delay (or
    * simply retry until the data is available).
    *
    * @return The names of the available search indexes.
    */
   @Named("search:indexes")
   @GET
   @Path("/search")
   @ResponseParser(ParseKeySetFromJson.class)
   @Fallback(EmptySetOnNotFoundOr404.class)
   Set<String> listSearchIndexes();

   /**
    * Searches all clients.
    * <p>
    * Note that without any request parameters this will return all of the data
    * within the index.
    *
    * @return The response contains the total number of rows that matched the
    *         request, the position this result set returns (useful for paging)
    *         and the rows themselves.
    */
   @Named("search:clients")
   @GET
   @Path("/search/client")
   @ResponseParser(ParseSearchClientsFromJson.class)
   SearchResult<? extends Client> searchClients();

   /**
    * Searches all clients that match the given options.
    *
    * @return The response contains the total number of rows that matched the
    *         request, the position this result set returns (useful for paging)
    *         and the rows themselves.
    */
   @Named("search:clients")
   @GET
   @Path("/search/client")
   @ResponseParser(ParseSearchClientsFromJson.class)
   SearchResult<? extends Client> searchClients(SearchOptions options);

   /**
    * Searches all items in a data bag.
    * <p>
    * Note that without any request parameters this will return all of the data
    * within the index.
    *
    * @return The response contains the total number of rows that matched the
    *         request, the position this result set returns (useful for paging)
    *         and the rows themselves.
    */
   @Named("search:databag")
   @GET
   @Path("/search/{databagName}")
   @ResponseParser(ParseSearchDatabagFromJson.class)
   SearchResult<? extends DatabagItem> searchDatabagItems(@PathParam("databagName") String databagName);

   /**
    * Searches all items in a data bag that match the given options.
    *
    * @return The response contains the total number of rows that matched the
    *         request, the position this result set returns (useful for paging)
    *         and the rows themselves.
    */
   @Named("search:databag")
   @GET
   @Path("/search/{databagName}")
   @ResponseParser(ParseSearchDatabagFromJson.class)
   SearchResult<? extends DatabagItem> searchDatabagItems(@PathParam("databagName") String databagName,
         SearchOptions options);

   /**
    * Searches all environments.
    * <p>
    * Note that without any request parameters this will return all of the data
    * within the index.
    *
    * @return The response contains the total number of rows that matched the
    *         request, the position this result set returns (useful for paging)
    *         and the rows themselves.
    */
   @SinceApiVersion("0.10.0")
   @Named("search:environments")
   @GET
   @Path("/search/environment")
   @ResponseParser(ParseSearchEnvironmentsFromJson.class)
   SearchResult<? extends Environment> searchEnvironments();

   /**
    * Searches all environments that match the given options.
    *
    * @return The response contains the total number of rows that matched the
    *         request, the position this result set returns (useful for paging)
    *         and the rows themselves.
    */
   @SinceApiVersion("0.10.0")
   @Named("search:environments")
   @GET
   @Path("/search/environment")
   @ResponseParser(ParseSearchEnvironmentsFromJson.class)
   SearchResult<? extends Environment> searchEnvironments(SearchOptions options);

   /**
    * Searches all nodes.
    * <p>
    * Note that without any request parameters this will return all of the data
    * within the index.
    *
    * @return The response contains the total number of rows that matched the
    *         request, the position this result set returns (useful for paging)
    *         and the rows themselves.
    */
   @Named("search:nodes")
   @GET
   @Path("/search/node")
   @ResponseParser(ParseSearchNodesFromJson.class)
   SearchResult<? extends Node> searchNodes();

   /**
    * Searches all nodes that match the given options.
    *
    * @return The response contains the total number of rows that matched the
    *         request, the position this result set returns (useful for paging)
    *         and the rows themselves.
    */
   @Named("search:nodes")
   @GET
   @Path("/search/node")
   @ResponseParser(ParseSearchNodesFromJson.class)
   SearchResult<? extends Node> searchNodes(SearchOptions options);

   /**
    * Searches all roles.
    * <p>
    * Note that without any request parameters this will return all of the data
    * within the index.
    *
    * @return The response contains the total number of rows that matched the
    *         request, the position this result set returns (useful for paging)
    *         and the rows themselves.
    */
   @Named("search:roles")
   @GET
   @Path("/search/role")
   @ResponseParser(ParseSearchRolesFromJson.class)
   SearchResult<? extends Role> searchRoles();

   /**
    * Searches all roles that match the given options.
    *
    * @return The response contains the total number of rows that matched the
    *         request, the position this result set returns (useful for paging)
    *         and the rows themselves.
    */
   @Named("search:roles")
   @GET
   @Path("/search/role")
   @ResponseParser(ParseSearchRolesFromJson.class)
   SearchResult<? extends Role> searchRoles(SearchOptions options);

}

redpajama_set_name: RedPajamaGithub


Zany Zygote Review accepts poetry, art, micro fiction, flash fiction, short fiction, novelettes, poetry chapbooks, and nonfiction. Please limit submissions to five per issue. An issue is one month. Poetry can be no longer than 30 lines.
Simultaneous submissions are allowed if you make it clear in your submission that your work is being considered elsewhere. We do not accept reprints unless the piece was published on a personal blog.


text: He is a French professional football coach and former player who is currently the manager of Major League Soccer club Montreal Impact. He is considered one of the greatest strikers of all time and has often been debated by pundits and journalists as the greatest striker in the history of the Premier League. In 2003 and 2004, Henry was the runner-up for the FIFA World Player of the Year and was runner-up for the Ballon d'Or in the former year. He was named the PFA Players' Player of the Year twice, the FWA Footballer of the Year three times, and was included in the UEFA Team of the Year five times. He was also one of the most commercially marketed footballers during the 2000s. Henry made his professional debut with Monaco in 1994 before signing for defending Serie A champions Juventus. However, limited playing time, as well as disagreements with the club's hierarchy, led to him signing for English club Arsenal for £11 million in 1999. Under long-time mentor and coach Arsène Wenger, Henry became a prolific striker and Arsenal's all-time leading scorer with 228 goals in all competitions. He won the Premier League Golden Boot a record four times, won two FA Cups and two Premier League titles with the club, including one during an unbeaten season. He spent his final two seasons with Arsenal as club captain, leading them to the 2006 UEFA Champions League Final. In June 2007, he transferred to Barcelona. In the 2008-09 season, he was a key part of the club's historic treble when they won La Liga, the Copa del Rey and the UEFA Champions League. In 2010, he joined New York Red Bulls of Major League Soccer(MLS). He returned to Arsenal on loan for two months in 2012, before retiring in 2014. Henry enjoyed sustained success with France, winning the 1998 FIFA World Cup UEFA Euro 2000 and 2003 FIFA Confederations Cup. He was awarded the French Player of the Year five times. He was also named to the UEFA Euro 2000 Team of the Tournament, awarded both the 2003 FIFA Confederations Cup Golden Ball and Golden Shoe and was named to the 2006 FIFA World Cup All-Star Team. In October 2007, he became his country's record goalscorer. After amassing 123 appearances and 51 goals, Henry retired from international football after the 2010 FIFA World Cup.


text:
\section{Introduction}
Federated Learning (FL) \cite{mcmahan2017communication} is a machine learning paradigm where user data is not shared for privacy protection.
The model is trained via arranged communications between the server and many system clients.
In each communication round, the selected clients run local optimization algorithms for multiple steps and send the local model updates back to the server.
The server aggregates the received local updates and then applies a gradient-based server optimizer, taking the aggregated update as the gradient input.

In recent years, deep neural network based models have proven to be successful in click-through rate (CTR) prediction~\cite{Song2019autoint, wang2021dcn}.
Based on user, item and context features, they predict CTR without tedious crafting of input features or higher-order interactions between them.

In this work, we focus on FL application for CTR prediction models.
Specifically, we propose an algorithm to learn the aggregation strategy of the local model updates in FL CTR models,
where the aggregation strategy is adjusted according to the feedback during the learning process.
There are two main motivations for our work described below.

\minisection{Weighting of local updates and client heterogeneity.}
Weighted averaging is a commonly-used strategy for local update aggregation in FL.
When all clients are homogeneous, taking the average over local updates gives good results \cite{li2019convergence}.
In practice, however, the clients are often heterogeneous.
This client heterogeneity is a major challenge in FL for CTR prediction models and recommender systems (RecSys) in general, because it causes inconsistencies between the client and the global objectives;
it slows down the training progress and often leads to suboptimal final results.
This issue is also known as \textit{client drift} \cite{karimireddy20scaffold}.

To obtain faster learning progress and better final solutions, a good client weighting strategy for update aggregation should be
(1) aware of the client heterogeneity: the importance weight of a client should depend on its attributes;
(2) adaptive: at different phases of the learning process, the strategy may focus on different sets of clients;
(3) parameter-wise: not all parameters behave in the same way during a training process, the strategy should support different learning dynamics for them.
For example, in FL of deep CTR models, there are parameters for sparse feature embeddings that are only updated by a small set of clients. On the contrary, dense fully-connected layers are updated by all clients in each round. The optimal weighting strategies for these two sets of parameters are likely to be different.

\minisection{Server learning rate tuning.}
Learning rate is arguably one of the most important hyper-parameters for gradient-based learning algorithms in centralised machine learning \cite{Bengio2012}.
Likewise, server learning rate, used by the server optimizer to control the step-size of the aggregated update, has an important role in FL~\cite{charles2020outsized, reddi2021adaptive, wang2021field}.
Similar to the case of client weighting, it is desirable that the server learning rate schedule is \textit{adaptive} and \textit{parameter-wise}.

In this paper, we propose an online meta-learning method to learn a strategy to aggregate the
local model updates in federated learning of CTR prediction models, which has the advantages
of being aware of client heterogeneity, adaptive to learning process, and parameter-wise.
Our contributions include:
\begin{itemize}
    \setlength\itemsep{0em}
    \item Meta-learning formulation of the aggregation problem in FL to
    compute the gradients of meta-parameters without the need of an extra dataset.
    \item A new online meta-learning method (MetaUA) that jointly learns the target CTR prediction model and the meta-aggregation model.
    \item Extensive experimental evaluations, including: comparison to other methods on
    four publicly available datasets, ablation study, analysis of clients' attributes selection,
    and method robustness to varying percentages of participating clients and meta-learning rate. MetaUA
    outperforms state-of-the-art methods in both the speed of convergence and the final results in few different
    settings.
\end{itemize}

\vspace{-2ex}
\section{Related work}
The first FL algorithm FedAvg was proposed ~\cite{mcmahan2017communication}.
Since then, there have been many FL works dedicated to RecSys \cite{fedfast, flanagan2020fedmvmf, lin2020metamf}.
While most of these works focus on learning generalized matrix factorization models \cite{He2017ncf},
our work is agnostic to model architectures.

Client heterogeneity is recognized as one of the key challenges in FL~\cite{wang2021field, Kairouz2021Advances}.
A few approaches have been proposed to address the client drift problem caused by heterogeneity:
FedNova~\cite{Wang2020fednova} normalizes the local updates before averaging to eliminate the objective inconsistency;
SCAFFOLD~\cite{karimireddy20scaffold} uses control variates to correct the drift in the local updates;
FedProx~\cite{Li2020Fedprox} adds a proximal term in the local objectives to stabilize the learning.

On the aggregation of local model updates, FedAvg uses a weighted average by the number of samples received from the clients.
Inspired by the attention mechanism \cite{Vaswani2017Attention, Ji2019FedAtt, fedfast} use the Euclidean distance between local and global models as the importance weights of clients.
In a similar spirit, \cite{chen2021dynamic} uses the divergence of local models to the global model as the importance of clients, but the importance is used in client sampling instead of aggregating updates. A similar work in client sampling with priority is discussed in \cite{Goetz2019Active}, where a local loss value is used as client importance.
On the adaption and scheduling of the learning rates in FL:
\cite{reddi2021adaptive} proposes adaptive optimization methods, such as Adagrad, Adam and YOGI, at the server side;
\cite{charles2020outsized} proposes a heuristic rule to adjust server and local learning rates based on loss values.
All these existing works on local model update aggregation and learning rate adaption are all based on manually engineered strategies, which often require a great amount of efforts in both designing and experimentation.

Our proposed algorithm belongs to the class of online meta-learning~\cite{Finn2019Online}.
In non-FL settings, ideas have be explored in the learning rate adaption and training example weighting:
gradient-based methods to learn the learning rate are proposed in \cite{gunes2018hypergrad}.
It is extended by \cite{Micaelli2020LongHorizons} from the next step objective to a long-horizon one.
On training example weighting, Meta-weight-net~\cite{Shu2019MetaWeightNet} learns the weight of training examples to address the bias and noise in the training data.
In this thread, \cite{Zhang2021NoReward} proposes a method to learn without the need of a reward dataset and
\cite{Xu2021famus} speeds up the meta-gradient computation with a faster layer-wise approximation.
In FL, \cite{chen2019federated} applies MAML~\cite{finn2017maml} and Meta-SGD~\cite{li2017metasgd} to learn the initial model weights for local training on each client.
However, the idea of using meta-learning to adapt the learning rate or client weights is yet to be explored in FL.

\vspace{-2ex}
\section{Problem formulation}
CTR prediction can be formulated as a supervised learning problem: given the input feature and click/no click label pairs $\{x, y\}$,
the goal is to learn a function $h(x; w)$ parameterized by $w$,
which minimizes the loss between the prediction and the ground truth label $\ell(h(x; w), y)$. At communication round $t$ of FL, a set of clients $S^t$ are selected for a participation, to whom a copy of
the global model $w$ is distributed.
Each  client $k \in S^t$ trains the model using their local data $D_k$
to generate a local model $w_k^t$:
$
w_k^t = \text{opt-local}(w^t; D_k).
$
$\text{opt-local}(\cdot)$ is the local optimization of $\ell(\cdot)$
initiated using weights $w^t$ on dataset $D_k$, which is often several epochs of SGD.
The clients send back to the server the model updates ${\Delta W}^t := \{ \Delta w_k^t, k \in S^t\}$,
where ${\Delta w}_k^t := w_k^t - w^t$.
These local updates are then aggregated at the server to get the update on the global model
$
\Delta w^t = \text{agg}({\Delta W}^t, Z^{t}).
$
where $Z^t := \{z_i, i \in S\}$ are the non-sensitive client attributes and $\text{agg}(\cdot)$ is the aggregation function which takes the local updates
and the client attributes as input and outputs the aggregated update.
The aggregated update $\Delta w^t$ is then applied to the
global model $w^t$ by performing one step of server optimization, i.e. $\text{opt-server}(\cdot)$, to get the new global model
 $ w^{t + 1} = \text{opt-server}(w^t, \Delta w^t).$

Many existing FL algorithms can be formulated under this framework. A few examples are:
\noindent{$\blacktriangleright$~FedAvg \cite{mcmahan2017communication}}
$
\text{agg}({\Delta W}^t, Z^t) = \frac{1}{n^t} \sum_{k \in S} n^t_k \cdot \Delta w^t_k,
$
and\\$\text{opt-server}(w, \Delta w) = w^t + \Delta w^t,$
where $z_k^t = n_k, \text{ for } k \in S^t$, $n_k$ is the number of samples for client $k$, and
$n^t = \sum_{k \in S^t} n_k$.

\noindent{$\blacktriangleright$~FedNova}\cite{Wang2020fednova}
$\text{agg}({\Delta W}^t, Z^t) =\big(\sum_{k \in S^t} \tau_k \cdot \frac{n_k}{n^t}\big) \cdot
\sum_{k \in S} \frac{n_k}{n^t} \cdot \frac{1}{\tau_k} \cdot \Delta w_k^t,
$
where $z_k^t = [n_k, \tau_k]$, $\tau_k$ is the number of local gradient descent steps on
client $k$.
The function $\text{opt-server}(\cdot)$ is the same as that of FedAvg.

\noindent{$\blacktriangleright$~FedAdam and FedAdagrad}\cite{reddi2021adaptive}
These two methods have the same aggregation function as FedAvg.
The server optimization method maintains first and second order momentum, $m^t$ and $M^t$:
$
  m^t = \beta_1 \cdot m^{t - 1} + (1 - \beta_1) \cdot \Delta w^t,
$
and
$
 M^t = M^{t - 1} + (\Delta w^t)^2, \space \text{ for FedAdagrad},
$
$
M^t = \beta_2 \cdot M^{t - 1} + (1 - \beta_2) \cdot (\Delta w^t)^2, \text{   for FedAdam},
$
The optimization step is performed as:
$
w^{t+1} = w^t + \gamma_s \cdot \frac{m^t}{\sqrt{M^t} + \epsilon},
$
where $\gamma_s, \beta_1, \beta_2 \text{ and } \epsilon$ are all hyper-parameters.
$\gamma_s$ is generally known as server learning rate.

\vspace{-2ex}
\section{Online Meta Learning for Local Update Aggregation}

To have a parameter-wise aggregation model, we will first define a partition of the weight indices $\mathcal{P}$, s.t. $\bigcup_{A \in \mathcal{P}} A$ is all the weight indices and
$A_1 \cap A_2 = \varnothing, \text{for any pair of } A_1, A_2 \in \mathcal{P}, A_1 \neq A_2$.
The most common partition is layerwise, where each $A \in \mathcal{P}$ is the set of weight indices of a network layer.
We will use $[\cdot]$ to denote the indexing operation on the weights.

\minisection{Meta model and its parameters} Our learnable aggregation model $\text{agg}(\cdot)$ is defined as:
\begin{equation*}\label{eq:agg_layer}
\begin{split}
    \Delta w^t[A] &= \text{agg}({\Delta W}^t[A], Z^t[A]; \theta[A]) := \theta_{s}[A] \cdot \sum_{k \in S^t} \alpha_k[A] \cdot \Delta w_k[A], \\
    & \text{ where } \alpha_k[A] = \text{softmax}(f_{\alpha}(z^t_k[A]; \theta_{\alpha}[A])),
     \forall A \in \mathcal{P}.
\end{split}
\end{equation*}
$\theta = \{\theta_s, \theta_{\alpha}\}$ are the meta parameters to be learned.
Each element of $\theta_s$ is between 0 and 1.
It is the scaling on the server learning rate for step-size adaption during the learning process.
$f_{\alpha}(z; \theta_{\alpha})$ is the client weighting function parameterized by $\theta_{\alpha}$, which gives the importance of the clients
based on their non-sensitive attribute(s) $z$.
$f_{\alpha}$ should be differentiable in $\theta_{\alpha}$.
The selection of $f_{\alpha}$ and $z^t_k$ will be detailed in Sections~\ref{sec:alg} and~\ref{sec:experiments}.
The output of the meta model is the aggregated update $\Delta w^t[A], A \in \mathcal{P}$, which is applied to get the new weight:

$w^{t+1}[A] = \text{opt-server}(w^t[A], \Delta w^t[A]), \forall A \in \mathcal{P}$.
This formulation allows the flexibility of aggregating different subsets of weight updates differently.
We will omit this indexing in the rest part of the paper for readability.

\minisection{Meta loss.}
Each local dataset $D_k$ is first split into
the \textit{support} dataset $D^{(s)}_k$ for local training,
and the \textit{query} dataset $D^{(q)}_k$ for meta loss evaluation.
The meta loss function is then defined in a \textit{delayed} way, i.e. the loss for the meta parameters at
communication round $t - 1$, $\theta^{t - 1}$ will be evaluated at the round $t$:
\begin{equation}\label{eq:mlossa}
L^t(\theta^{t - 1})  = \sum_{k \in S^t}  L^t(\theta^{t - 1};  D_k^{(q)}),
\end{equation}
where each

$L^t(\theta^{t - 1}; D_k^{(q)}) := $ $\sum_{\{x, y\} \in D_k^{(q)}}  \ell\big(h(x; w^{t}); y\big)$, abbreviated as $L^t_k(\theta^{t - 1})$.
The motivation is that, at the current round $t$, we evaluate the quality of our aggregation
strategy performed at the previous round $t-1$ and adjust it accordingly.
One advantage of (\ref{eq:mlossa}) that it is an unbiased estimation
of the training loss of $w^{t}$ from training examples on $S^t$.
Another advantage is that both its evaluation and optimization is done \textit{online}; this means that no extra datasets are needed nor are additional communication rounds required.

\minisection{Meta optimization}
We use gradient descent to optimize the meta parameters. The computation of the meta gradient is one of the main
contributions of this paper.
This section gives the description on the computation of $\frac{\partial L^t}{\partial \theta^{t - 1}} \big(\theta^{t - 1}\big)$. We will slightly abuse the notation by using $\theta^{t - 1}$ for both the variable and its value, which can be distinguished by the context.

Similar to the meta loss, its gradient is also computed in a \textit{delayed} way: the gradient of
$\theta^{t - 1}$ is computed at round $t$.
To do this, we will need to store the model weight, local model updates and clients' attributes of the previous round at the server,
so that we can assume that at round $t > 1$, we have
$w^{t-1}, \Delta{w}^{t - 1}_k \text{ and } z^{t-1}_{k}, \forall k \in S^{t - 1}$ avaialble.

By the chain rule, we have
\begin{equation}\label{eq:mgd}
\begin{split}
    \frac{\partial L^t}{\partial \theta^{t - 1}} \big(\theta^{t - 1}\big) =
    \frac{\partial L^t} {\partial w^t}\big(w^t\big)
    \cdot \frac{\partial w^{t}}{\partial \theta^{t - 1}}\big(\theta^{t-1}\big).
\end{split}
\end{equation}
The first part of \eqref{eq:mgd} RHS, denoted as $g^t$, is the gradient of $L^t$ w.r.t $w^t$ evaluated at $w^t$ on the training data of the clients in $S^t$, i.e.

$
 g^t = \sum_{k \in S^t }\underbrace{\frac{\partial L^t_k} {\partial w^t}\big(w^t\big)}_{g^t_k := }.
$
Each $g^t_k$ is evaluated locally on each client $k \in S^t$ as
\begin{equation}\label{eq:gtk}
g^t_k = \sum_{x,y \in D_k^{(q)}} \frac{\partial \ell} {\partial w^t}\big(f(x, w^t), y\big),
\end{equation}
and then sent back to the server.
The second part of \eqref{eq:mgd} RHS is the Jacobian matrix
${\partial w^{t}}/{\partial \theta^{t - 1}}$ evaluated at $\theta^{t-1}$.
The dependency of $w^{t}$ on $\theta^{t - 1}$ is through $\Delta w^{t}$ as the composite of the two functions $\text{agg}(\cdot)$ and
$\text{opt-server}(\cdot)$.
Both of these two functions are differentiable, so we can also apply the chain rule:
$
    \frac{\partial w^{t}}{\partial \theta^{t - 1}} = \frac{\partial w^{t}}{\partial \Delta w^{t-1}} \cdot
    \frac{\partial \Delta w^{t-1}} {\partial \theta^{t - 1}}.
$
In the implementation, there is no need to compute the full Jacobian matrices.
Instead, since $g^t$ is not a function, we move it into the second paritial derivative in Eq.~\eqref{eq:mgd}, which becomes:
\begin{equation}\label{eq:vjp}
 \frac{\partial L^t}{\partial \theta^{t - 1}}\big(\theta^{t-1}\big) = \frac{\partial (g^t \cdot w^t)}{\partial \theta^{t - 1}}
 \big(\theta^{t-1}\big) .
\end{equation}
The inner product $g^t\cdot w^t$ is a scalar-output linear function of $w^t$. Its gradient w.r.t $\theta^{t - 1}$ can
easily be computed through back propagation provided by most automatic differentiation packages. The procedure of gradient
computation in MetaUA in two FL rounds is presented in Fig.~\ref{fig:metaua}.

\begin{figure}[t]
    \centering
    \includegraphics[scale=0.5]{figs/idea.png}
    \caption{Overview of the gradient computation procedure in MetaUA: \textcircled{1} local model updates and attributes are sent from selected clients; \textcircled{2} local updates are aggregated using meta model; \textcircled{3} new model is distributed to the selected clients;
    \textcircled{4} server receives the gradients from the selected clients and computes
    the gradients for the meta-parameters. }
    \label{fig:metaua}
\end{figure}

\minisection{MetaUA Algorithm}
\label{sec:alg}
The proposed algorithm of Online \textbf{Meta}-Learning~for \textbf{U}pdate \textbf{A}gg\-regation (\textbf{MetaUA}) is given in Algorithm~\ref{alg}, where all computations follow previously described derivations.
The key difference between it and the other FL algorithms is that, at each communication round,
the meta parameters will be updated by a step of gradient descent, which are then used to aggregate local updates to form a new version of the server's model.

\SetAlCapNameFnt{\footnotesize}
\SetAlCapFnt{\footnotesize}
\begin{algorithm}
\footnotesize
\DontPrintSemicolon

    \KwInput{ Randomly initialized weight $w^1$}
    \KwOutput{ $w$ after the learning is complete}
    \KwData{ An universe of clients $I$, for each client $k \in I$,
    an associated local dataset $D_k = D_k^{(s)} \cup D_k^{(q)}$}

    \For{communication round t = 1, 2, \dots, T} {
        Randomly select a set of clients $S^t$ \\
        \For { each client $k$ in $S^t$ }{
            $\Delta w^t_k \leftarrow \text{opt-local}(w^t, D_k^{(s)}) - w^t$  \tcp*{local training}
            compute $g^t_k$ with equation (\ref{eq:gtk}) \\
            set client attributes to $z^t_k$ \\
        }

        \If{  $t > 1$ } {
            \tcc{backward pass of update aggregation}
            $g^t \leftarrow \sum_{k \in S^t} g^t_k$ \\
            compute $\frac{\partial L^t}{\partial \theta^{t - 1}}\big(\theta^{t - 1}\big)$ with equation (\ref{eq:vjp}) \\
            $\theta^t \leftarrow \theta^{t - 1} - \gamma_{meta} \cdot \frac{\partial L^t}{\partial \theta^{t - 1}}\big(\theta^{t - 1}\big) $
        }

        \tcc{forward pass of update aggregation}
        $\Delta w^t \leftarrow \text{agg}(\Delta W^t, Z^t; \theta^t)$  \\
        $w^{t + 1} \leftarrow \text{opt-server}(w^t, \Delta w^t)$
    }
\caption{Online Meta-Learning for Update Aggregation (\textbf{MetaUA})}
\label{alg}
\end{algorithm}

\vspace{-1.5ex}
\paragraph{Server optimizer}
Our algorithm is flexible in the selection of the server optimizer.
We run experiments on SGD, FedAdagrad and FedAdam, in which FedAdagrad has the best results.
We thus choose to use FedAdagrad in all the experiments in this paper.

\vspace{-1.5ex}
\paragraph{Client weighting model}
\label{sec:metaua_z}
The main consideration in choosing $f_{\alpha}$ is that it will run for all the selected clients in both forward
and backward pass, therefore, the use of a simple architecture to keep the additional computations
small should be favored.
In our experiments, we found that using a simple linear model for $f_{\alpha}$ is enough to get
good results.

For client attributes, a good candidate should:
1) not reveal user privacy;
2) capture the client heterogeneity;
3) introduce little or no extra computation.
Inspired by \cite{Shu2019MetaWeightNet} and \cite{Goetz2019Active}, we choose the local loss
$z^t_k = \frac{1}{n_k}\sum_{\{x, y\} \in D_k} \ell(f(x; w^t); y)$.
Intuitively, the local loss measures how well the current model fits the client's local data.
A higher local loss than others could be caused by either system or data heterogeneity.
Some possible reasons are:
the client has lagged behind others in learning because it has not actively participated in FL,
or less local training steps have been performed on it because it has less powerful hardware,
or the client has examples that the current model fits poorly,
which could be examples novel to the model, ambiguous examples near the decision boundary or mislabelled  examples.
We have also considered including: number of samples,norm of the weight updates, number of unique features, etc.
Experimental results are provided in section~\ref{sec:experiments}.

$\gamma_{meta}$ is the learning rate on the meta-parameters. We find that our method is generally robust to it. In our experiments, we fix it to be $0.1$, which has reasonably good results under all the settings.

\vspace{-2ex}
\section{Cost Analysis}
Compared to other FL methods there are some extra costs incurred from MetaUA at each training round.

\minisection{Communication}
To compute $g^t$, we need to transfer the gradient of the local loss w.r.t the model parameter to the server for all the selected clients, i.e. $g^t_k, k \in S^t$.
This will double the communication cost from the clients to the server.
A few methods can be explored to reduce the communication: compressing the model gradient before sending or sub-sampling the clients in gradient evaluation.

\minisection{Computation} On the client side, the extra computation needed is for $g^t_k, k \in S^t$.
This requires one additional pass over the local data. We consider the increase in computation to be marginal, as each selected client already have to perform several epochs of training in FL round.
On the server side, an extra computation is needed for: the meta aggregation operator, the opt-server, the inner product in \eqref{eq:vjp} and a backward pass on them for the gradient computation.
The aggregation operator involves shallow networks to be applied as many times as the number of selected clients in an FL round, while other operations are just a constant number of simple operations.
The impact of this extra computation cost on the server side is not significant, because the server is usually a high-performance cluster.

\minisection{Storage} Since our meta gradient evaluation is delayed, we need to store the model updates received from the selected clients for a single communication round. In practice, the user population is quite large, so additional memory resources have to be allocated for that.

\vspace{-2ex}
\section{Experiments}
\label{sec:experiments}
We evaluated a proposed method for a click-through rate (CTR) prediction task in FL setting that can be used in
real-world scenario. A series of experiments are designed and conducted to present methods properties.

\minisection{Datasets}
In our experiments we use four well-know datasets: MovieLens-1M, Tmall, Yelp, and Amazon-Cds.
Their main statistics are presented in Table~\ref{tab:datasets}.
Input features can be related to: user, item or context of the event. Data of each user is taken as
a client's partition in all federated learning experiments. The last 10\% of client's data (based on a timestamp)
is taken for a validation.

\begin{table}[tb]
\centering
\resizebox{0.97\columnwidth}{!}{%
\begin{tabular}{lrrrrrr}
\toprule
Dataset &  \# Examples & \# Features  &  Vocab Size  & \# Users &  \# Items &  Density \\
\midrule
MovieLens-1M &  739012 &  7 & 13196 & 6040 & 3668 & 3.34 \\
Tmall & 1899378 & 4 & 44239 &  22284 & 17705 & 0.48 \\
Yelp & 530124 & 9 &  34462 & 22128 & 12232 &  0.20\\
Amazon-CDs & 890824 & 3 &  31985 & 15592 & 16184 &   0.35\\
\bottomrule
\end{tabular}
}
\caption{Dataset statistics}
\label{tab:datasets}
\end{table}

\minisection{Implementation Details} In all experiments we use DCNv2~\cite{wang2021dcn} prediction model with binary
cross entropy as a loss function and dimension for all embeddings set to $4$. For a
central training experiments Adam optimizer is used for ten epochs with
$lr=0.0001$, weight decay $0.0001$, batch size of $256$. In FL settings we use SGD as a local optimizer
with $lr=0.01$ with batch size of $15$ and three epochs done in each of 200 communication rounds. In proposed method,
meta-parameters are optimized with SGD with $lr=2.0$. We use layer-wise approach where each parts of the network
have dedicated meta-parameters. Implementation is done using PyTorch library.

\minisection{Comparison with other methods} The results for comparison of MetaUA with FedAvg and FedAdagrad are presented in Table~\ref{tab:methods_cmp}.
Both adaptive methods are better than simple FedAvg, they converge faster, and the training process is more stable.
MetaUA got slightly better final AUC than FedAdagrad or very competitive (AmazonCDs worse only by $0.007$).
However, if we consider for those results logloss value during the train the difference is big. On average MetaUA
gets $8.5\%$ better logloss. This margin is noticeable in FL training curves
in Figure~\ref{fig:cmp_plots}.

\begin{figure*}[t]
\centering
\includegraphics[width=0.8\textwidth,keepaspectratio]{figs/fl_cmp.pdf}
\vspace*{-3ex}
\caption{AUC and Logloss values during the federated train with methods FedAvg, FedAdaptive and MetaUA for
MovieLens-1M and Tmall datasets.}
\label{fig:cmp_plots}
\end{figure*}

\begin{table}[tb]
\centering

\resizebox{\columnwidth}{!}{%
    \input{tab_cmp.tex}
}
\caption{Performance evaluation of FedAvg, FedAdagrad and MetaUA over different datasets. For a comparison
logloss and AUC are presented for different rounds of federated learning. Best values are in bold.}
\label{tab:methods_cmp}
\end{table}

\minisection{Selection of client's attributes}
We evaluated a few different information that can be shared to a server from clients during a FL training.
Following desired characteristic described in section~\ref{sec:metaua_z} we used:
$z_1$ - number of samples~\cite{mcmahan2017communication},
$z_2$ - local loss value~\cite{Shu2019MetaWeightNet},
$z_3$ - gradient norm,
$z_4$ - ratio of loss value after local SGD train to local loss,
$z_5$ - positive class probability,
and $z_6$ - number of unique features in local dataset~\cite{Wistuba2016TwoStageTS,Wistuba2017ScalableGP,Jomaa2019HypRLH}.
Each of the variables grasps a different aspect of a client's local dataset and local training process during
the FL communication round. Sampled clients $Z$ distributions during FL train for MoveieLens-1M and Amazon-CDs datasets
are presented in Figure~\ref{fig:metaua_z_dist_1}.
The smallest variation presents $z_3$. However, this is the only one dependent on FL round as well as the
layer of the network (computed layer-wise, on plots we see aggregated values).

Table~\ref{tab:metaua_z_amz} present the results for Amazon-CDs dataset.
The best final logloss value is achieved only using $z_2$ (local loss), which happens to be
as well a good proxy for client importance in previous work~\cite{Shu2019MetaWeightNet}. Up to some point in
FL training, using only $\theta_s$ is also beneficial. $z_6$ (unique features num.) and $z_2$ are second good choices.
Surprisingly, taking all variables does not yield the best results. For evaluated datasets it seems like
simpler choices and models works better with an online meta-learning. In our experiments we evaluated
different models, i.e.~MLP network for $f_{\alpha}(Z; \theta_{\alpha})$. However, the results for simple linear
model were better.

\begin{table}[tb]
\centering
\resizebox{\columnwidth}{!}{%
    \input{tab_metaua_z_amazon.tex}
}
\caption{Performance of MetaUA evaluated with different set of client's attributes $Z$ on Amazon-CDs dataset.}
\label{tab:metaua_z_amz}
\end{table}

\begin{figure}[tb]
\centering
\subfigure{\includegraphics[width=0.2\textwidth,keepaspectratio]{figs/dist_z_ml1m_epoch_20.pdf}}
\subfigure{\includegraphics[width=0.2\textwidth,keepaspectratio]{figs/dist_z_amazon-dien-cds-10_epoch_20.pdf}}
\caption{Distribution of $Z$ variables for datasets Movielens-1M (left) and Amazon-CDs (right).
Snapshot of sampled values are taken during round number 20.}
\label{fig:metaua_z_dist_1}
\end{figure}

\minisection{Ablation study} In order to evaluate contribution of different elements of MetaUA method we performed ablation study where each
part of the approach is disabled. Four versions are evaluated in very same setting: only server learning rate
meta-learning, client weighting with meta-learning, both at the same time and no adaptation at all.
Comparison is presented in Table~\ref{tab:ablation} for MovieLens-1M dataset.
While the model is optimized toward the best logloss value, here application of whole MetaUA gives the best advantage.
For a longer FL training period, at round 400, adaptation of server learning rate and client weighting yield
similar outcome of $0.413$. Only the application both at once lower the results to $0.382$. AUC seems also
beneficial when all elements of MetaUA are used.

\begin{table}[tb]
\centering
\resizebox{\columnwidth}{!}{%
    \input{tab_ablation.tex}
}
\caption{Ablation study}
\label{tab:ablation}
\end{table}

\minisection{Different fraction of clients} To present robustness of MetaUA towards different fraction of client sampled during FL training rounds, we
lowered this value to $5\%$ and $1\%$. Comparison with FedAdagrad is presented in Figure~\ref{fig:fl_frac_1}.
We presented a longer training as with fewer clients during training more rounds can be necessary for the convergence.
While fraction get smaller, MetaUA still wins with FedAdagrad with a big margin. For MovieLens-1M, the difference
is clear. Only FedAdagrad with fraction $5\%$ outperforms $10\%$ with more than 350 rounds. For Amazon-CDs
the best at final round got MetaUA with $5\%$ clients sampled in each round.

\begin{figure}[tb]
\centering
\subfigure{\includegraphics[width=0.8\linewidth,keepaspectratio]{figs/fl_frac_ml1m.pdf}}\\[-3ex]
\subfigure{\includegraphics[width=0.8\linewidth,keepaspectratio]{figs/fl_frac_amazon.pdf}}
\vspace*{-1.7em}
\caption{Comparison of MetaUA and FedAdagrad training curves when different fraction of clients is used during FL rounds.}
\label{fig:fl_frac_1}
\end{figure}

\minisection{Robusness to hyper-params - selection of meta lr}
MetaUA is helping to solve its main task -- adaptive optimization during FL on the server side.
However, as any other meta-learning method, it introduces its own hyper-parameters at the meta-level.
As long as those are easy to find the whole process is applicable. For MetaUA the meta-learning rate is the one
most crucial. In all experiments we used fixed value of $2.0$, without doing extensive hyper-parameter search
each time. Thus, in Figure~\ref{fig:metaua_lr_ml1m}
we can see the performance for different values of the meta-learning rate on MovieLens-1M dataset in order
to present the robustness of our approach. It is easy to find the best performing meta-learning. Here,
for values bigger than one almost the same logloss at round 200 is received.

\begin{figure}[tb]
\centering
\includegraphics[width=0.8\linewidth,keepaspectratio]{figs/metaua_lr_ml1m.pdf}
\vspace*{-1.7em}
\caption{AUC and Logloss values during FL training on MovieLens-1M dataset for a various meta-learning rate value.
Logloss (right) plot is zoomed-in after 100 rounds to better present the difference.
}
\label{fig:metaua_lr_ml1m}
\end{figure}

\vspace{-2ex}
\section{Conclusions}
This paper presents a method of learning adaptive models for client weighting and server learning rate adaption in the aggregation of local model updates for federated deep CTR models.
In our experiments on public benchmarks, our method shows better performance compared to the state-of-the-art methods. The method is adjusted in an online way fitted in FL communication rounds, without a need of additional heavy communication from the clients.
As future work, one important direction it to improve the privacy protection of our method, for example, by introducing a random noise to both model updates and gradients communicated back to the server.
Another extension of a proposed MetaUA method is to learn the sample weighting for local trains in the FL settings.

{\footnotesize
\bibliographystyle{ACM-Reference-Format}

redpajama_set_name: RedPajamaArXiv