text:
Q: Need to restrict the access for staff users to view and edit only their inputted data in django admin I need to restrict the access in the admin page like if a staff user is logging in the admin they need to see only their entry data's and can able to add or delete their data in the admin panel.
As I tried the below code in the admin.py still I could able to view and delete everyone's data.
Admin.py

admin.site.site_header = "Inhouse Project Management"
admin.site.site_title = "TimeSheet Admin Portal"
admin.site.index_title = "TimeSheet Administration"

TokenAdmin.raw_id_fields = ['user']

class MyModelAdmin(admin.ModelAdmin):
    def get_queryset(self, request):
        qs = super().get_queryset(request)
        if request.user.is_superuser:
            return qs
        return qs.filter(user=request.user)

#Admin SuperUser
@admin.register(User)

class UserAdmin(ImportExportModelAdmin):
    list_display = ('id','employee_name','email','billable_and_non_billable','designation')
    search_fields = ['employee_name']
    pass

@admin.register(Project)

class ProjectAdmin(ImportExportModelAdmin):
    list_display = ('id','project_name','client','start_date','end_date')
    search_fields = ['project_name']
    pass

Urls.py
urlpatterns = [
    path('', TemplateView.as_view(template_name="social_app/index.html")), #social_app/index.html
    path('admin/', admin.site.urls),         #admin api
    path('api/',include(router.urls)),          #api
    path('accounts/', include('allauth.urls')),       #allauth
    re_path('rest-auth/', include('rest_auth.urls')),    #rest_auth
    path('api-auth/', include('rest_framework.urls')),
    re_path('rest-auth/registration/', include('rest_auth.registration.urls')),
    #path('api-token-auth/', views.obtain_auth_token),
    #path('api-token-auth/',CustomAuthToken.as_view()),
    #path('api/token/', TokenObtainPairView.as_view(), name='token_obtain_pair'),
    path('auth/login/',TokenObtainPairView.as_view(), name='token_obtain_pair'),
    path('api/token/refresh/', TokenRefreshView.as_view(), name='token_refresh'),
    path('conso/',view),
    path('chaining/', include('smart_selects.urls')),
]

models.py
class User(models.Model):

    BLOOD_GROUP_CHOICES = (

        ('a+','A+'),
        ('a-','A-'),
        ('b+','B+'),
        ('b-','B-'),
        ('ab+','AB+'),
        ('ab-','AB-'),
        ('o+','O+'),
        ('o-','O-')

    )

    BILLABLE_and_NON_BILLABLE_CHOICES=(

        ('Billable','Billable'),
        ('Non-Billable','Non-Billable')
    )

    employee_name = models.CharField(max_length=50)
    dob=models.DateField(max_length=8)
    email=models.EmailField(max_length=254,default=None)
    pancard=models.CharField(max_length=25,default=None)
    aadhar=models.CharField(max_length=20,default=None)
    personal_email_id=models.EmailField(max_length=254,default=None)
    phone = PhoneField(blank=True)
    emergency_contact_no=models.IntegerField(default=None)
    emergency_contact_name=models.CharField(max_length=100,null=True)
    relation=models.CharField(max_length=25,default=None)
    blood_group=models.CharField(max_length=25,choices=BLOOD_GROUP_CHOICES,null=True)
    designation=models.ForeignKey(Designation,on_delete=CASCADE,related_name="designation")
    billable_and_non_billable=models.CharField(max_length=25,choices=BILLABLE_and_NON_BILLABLE_CHOICES,default='Billable')
    joining_date=models.DateField(max_length=15,null=True)
    relieving_date=models.DateField(max_length=15,null=True)

    class Meta:
        db_table ='User'

    def __str__(self):
        return self.employee_name

serializers.py
class UserSerializers(serializers.ModelSerializer):

    #Email Notification
    def create(self, validate_data):
        subject = 'Your account was activated'
        plain_message = 'SAMPLE TEXT'
        from_email = 'demomaster147@gmail.com'
        to_email = validate_data['email']
        EmailThread(subject, plain_message, from_email, to_email).start()
        return User.objects.create(**validate_data)

I need to restrict the access for users to view and edit only their data in the admin panel. I dont know what needs to be updated to get that worked. Kindly help to resolve this issue.
Updated code as per the answer
from django.contrib.auth.admin import UserAdmin as BaseUserAdmin

class UserAdmin(BaseUserAdmin):
    def get_queryset(self, request):
        qs = super().get_queryset(request)
        user = request.user
        return qs if user.is_staff else qs.filter(user=user)

'''class MyModelAdmin(admin.ModelAdmin):
    def get_queryset(self, request):
        qs = super().get_queryset(request)
        if self.request.user.is_staff:
            return qs
        print(request.user)
        #return qs.filter(user=request.user)'''

#Admin SuperUser
@admin.register(User)

class UserAdmin(ImportExportModelAdmin):
    list_display = ('id','employee_name','email','billable_and_non_billable','designation')
    search_fields = ['employee_name']
    pass

Job model where the employee_name is linked under user field from the user model
class Job(models.Model):

    job_name=models.CharField(max_length=50)
    client=models.ForeignKey(Client,on_delete=CASCADE,related_name='client',default=None)
    project = ChainedForeignKey(Project,chained_field="client", chained_model_field="client",show_all=False, auto_choose=True, sort=True)
    user=models.ForeignKey(User,on_delete=CASCADE,related_name='user',default=None)
    hours=models.TimeField(null=True)
    start_date = models.DateTimeField(max_length=10)
    end_date=models.DateTimeField(max_length=10)

A: First you've to create a class that will handle displaying objects to respective user
class StaffAdmin(admin.ModelAdmin):

    def get_queryset(self, request):
        qs = super().get_queryset(request)
        if request.user.is_superuser:
            return qs
        return qs.filter(user=request.user)

then you've to register this class with your model where you want to add this functionality eg. if you want to display project of perticular user you've to register this class with Project model like this
admin.site.register(Project, StaffAdmin)

Note: If you've logged in as staff make sure you've given permission to your staff to (view, add, update, delete) projects or it will raise PermissionDenied error check how to add permissions to user. You may want to add user's permission programaticlly then check default-permissions[Django-doc]

A: Create a custom user model, i will put my code hope that help!
Crm views
from django.shortcuts import reverse
from .models import Lead
from .leadforms import LeadForm, CreatingUser
from django.views import generic
from django.core.mail import send_mail
from django.contrib.auth.mixins import LoginRequiredMixin

# Create your views here.
class signup(LoginRequiredMixin, generic.CreateView):
    template_name = 'registration/signup.html'
    form_class = CreatingUser

    def get_success_url(self):
        return reverse("LogInPage")

class leads(LoginRequiredMixin, generic.ListView):
    template_name = "crm/leads.html"
    queryset = Lead.objects.all()
    context_object_name = "leads"

class leadetails(LoginRequiredMixin, generic.DetailView):
    template_name = 'crm/leadetails.html'
    queryset = Lead.objects.all()
    context_object_name = "lead"

class leadcreate(LoginRequiredMixin, generic.CreateView):
    template_name = 'crm/leadcreate.html'
    form_class = LeadForm

    def get_success_url(self):
        return reverse("LeadsPage")

    def form_valid(self, form):
        send_mail(
            subject = "A lead Has been Created!",
            message = f"Agent is :{requset.user.username}",
            from_email = "Test@test.com",
            recipient_list= ["Tesy2@test.com"]
        )
        return super(leadcreate, self).form_valid(form)

class leadupdate(LoginRequiredMixin, generic.UpdateView):
    template_name = 'crm/leadupdate.html'
    form_class = LeadForm
    queryset = Lead.objects.all()

    def get_success_url(self):
        return reverse("LeadsPage")

class leadelete(LoginRequiredMixin, generic.DeleteView):
    template_name = 'crm/leadelete.html'
    queryset = Lead.objects.all()

    def get_success_url(self):
        return reverse("LeadsPage")

crm urls
from django.urls import path
from . import views
from django.contrib.auth.views import LoginView, LogoutView

urlpatterns = [
    path('', LoginView.as_view(), name="LogInPage"),
    path('logout', LogoutView.as_view(), name="LogOutPage"),
    path('signup', views.signup.as_view(), name="SignUpPage"),
    path('leads', views.leads.as_view(), name="LeadsPage"),
    path('leads/<int:pk>', views.leadetails.as_view(), name="Leadetails"),
    path('leads-create', views.leadcreate.as_view(), name="LeadCreate"),
    path('leads/<int:pk>-update', views.leadupdate.as_view(), name="Leadupdate"),
    path('leads/<int:pk>-delete', views.leadelete.as_view(), name="Leadelete"),
]

crm models
from django.db import models
from django.contrib.auth.models import AbstractUser

# Create your models here.
class User(AbstractUser):
    is_organisor = models.BooleanField(default=True)
    is_agent = models.BooleanField(default=False)

class UserProfile(models.Model):
    user = models.OneToOneField(User, on_delete=models.CASCADE)

    def __str__(self):
        return self.user.username

class Agent(models.Model):
    user = models.OneToOneField(User, on_delete=models.CASCADE)
    organisation = models.ForeignKey(UserProfile, on_delete=models.CASCADE)

    def __str__(self):
        return self.user.username

class Lead(models.Model):
    publish_date = models.DateTimeField(auto_now_add=True)
    full_name = models.CharField(max_length=50)
    passport_no = models.CharField(max_length=11)
    email = models.CharField(max_length=90)
    number = models.CharField(max_length=15)
    agent = models.ForeignKey(Agent, null=True, blank=True,  on_delete=models.CASCADE)

    def __str__(self):
        return self.full_name

def UserCreateSignal(sender, instance, created, **kwargs):
    if created:
        UserProfile.objects.create(user=instance)

models.signals.post_save.connect(UserCreateSignal, sender=User)

Crm form
from django import forms
from .models import Lead
from django.contrib.auth.forms import UserCreationForm, UsernameField
from django.contrib.auth import get_user_model

# Create your forms hear.

User = get_user_model()

class LeadForm(forms.ModelForm):
    class Meta:
        model = Lead
        fields = (
            'full_name',
            'passport_no',
            'email',
            'number',
            'agent',
        )

class CreatingUser(UserCreationForm):
    class Meta:
        model = User
        fields = ("username",)
        field_classes = {'username': UsernameField}

Base html

<!DOCTYPE html>

{% load static %}

    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>{% block title %} {% endblock %}</title>
    <link rel="shortcut icon" href="{% static 'crm/images/favicon.ico' %}" type="image/x-icon">
    <link rel="stylesheet" href="{% static 'crm/css/styles.css' %}">

    {% if not request.user.is_authenticated %}
    {% else %}
    {% if request.user.is_organisor %}
    Create a new user
    Agents
    {% endif %}
    Leads
    {% endif %}
    {% if request.user.is_authenticated %}
    Welcome {{ request.user.username }}
    Logout
    {% else %}
    Login
    {% endif %}
    {% block body %} {% endblock %}

you can assign different roles like is_organisor, is_agent in above code and you can restrict them from accessing and can edit in django admin
Reference images
Permissions
2nd permission ref
Note: don't copy and past it's an incomplete code


text:
Why Google ChromeCast and Apple TV were kicked off of Amazon
November 1, 2015 By Sam Catherman
The battle for dominance in the set-top box market continues as Apple TV and Google Chromecast were dropped from Amazon.
Competition is beginning to grow fierce in the market for streaming media hardware. Apple has announced its new version of the set-top box Apple TV, and Amazon responded in earnest by dropping both Apple TV and competitor Google ChromeCast devices from its online stores. According to a report from USA Today, the company seeks to avoid confusion to potential customers interested in their own streaming media service, Prime Video.
Nearly 50 percent of American households currently subscribe to an online streaming service like Netflix, Amazon Prime Video, or Hulu, and the number of people watching traditional cable television is dwindling fast. Analysts predict that as early as next year streaming television will begin to eclipse cable as viewer preferences shift.
Apple TV and ChromeCast are both projected to experience robust growth in the coming years as people purchase hardware that allows them to more easily access streaming content from their homes. Networks and content providers have already been teaming up to meet the new demand for fresh programming on streaming services, and analysts think this will be the main driver of competition between providers over the coming years.
Apple TV boasts connectivity with other Apple devices, which makes it an attractive option for many consumers. Prime Video comes with a whole list of other benefits associated with the Amazon Prime service, including free two-day shipping and access to original Amazon programming.
ChromeCast is the economic option for people looking to plug in and access content right away. A smaller competitor, Roku, is expected to make great gains in the coming years as well.
While there is already a long list of streaming devices consumers can choose from, the ultimate driver of competition between companies will be their ability to offer the most compelling original content. In the meantime, Amazon has had to resort to measures like removing Apple TV and ChromeCast from its offerings in order to direct attention towards Prime Video.
Amazon recently announced a multi-year deal with CBS to license exclusive content for Prime Video. A press release outlining the details of the deal can be found here.
Filed Under: Front Page, Technology

redpajama_set_name: RedPajamaCommonCrawl


Did you ever wonder why the banks and the City (who caused the great recession) were also the largest beneficiaries of re-inflating the economy with quantitative easing? And did you not perhaps feel a little aggrieved that, once they'd got what they wanted, they carried on as if nothing had happened, displayed no gratitude whatsoever and changed their ways not a bit? Anyway, history has shown that there is alternative in the form of a massive public works programme. You know, like the New Deal. A response to an economic calamity that benefits everyone, not just the people who caused it. Imagine if, instead of a few new towers in the City we had new jobs, new ports, new roads and new railways. Imagine, if instead of austerity, we had government spending. You wouldn't even need to print money to do it. Rather, governments could borrow it - and at the lowest rates for decades. The Economist recently described the failure of governments to borrow to build as a "missed opportunity", while Standard & Poor's reckons that, for every 1% of GDP spent on infrastructure the UK's economy would grow by 2.5 per cent.
Standard & Poor's sees clear economic benefits to G20 countries' increased public spending on infrastructure–with the so-called "multiplier effect" of an increase in spending of 1% of real GDP running as high as 2.5 in a three-year period. The multiplier effect is generally greater in developing economies than for more developed countries; for example, China, India, and Brazil would all enjoy a boost to GDP of at least double the increase in investment. For Europe, it's clear that a concerted effort across the region would have a greater effect than country-specific increases in spending. For developed nations, the increase would boost employment substantially–adding more than 700,000 jobs in the U.S. and about a million in the EU. In addition to the short-term boost to jobs and aggregate demand, infrastructure investment often yields long-term benefits by enhancing efficiency.


Connect your SuperSpeed USB 3.0 devices using this Ultra Slim Series USB 3.0 A Male to Micro B Male cable.
This cable features a USB 3.0 A male connector on one end and a USB 3.0 micro B male connector on the other end. The USB 3.0 micro B connector differs from the traditional USB 2.0 micro B connector in that it has an additional 5-pin plug, which carries power for external devices, such as portable hard drives.
It has an ultra slim design, which results in smaller and slimmer connector heads than ordinary cables. The A male connector head measures about 16mm wide, 8mm thick, and 20mm long. The micro B male connector head measures about 16mm wide, 7mm thick, and 20mm long.


Obtain the GEDCOM file in new tab from that website (free). Then search ($) Archives for Giles Badlesmere.
Obtain the GEDCOM file in new tab from that website (free). Then search ($) Archives for Giles De BADLESMERE.
Obtain the GEDCOM file in new tab from that website (free). Then search ($) Archives for Giles Lord Badlesmere.
Obtain the GEDCOM file in new tab from that website (free). Then search ($) Archives for John Ferrers.
Obtain the GEDCOM file in new tab from that website (free). Then search ($) Archives for Henry FOWLER.
Obtain the GEDCOM file in new tab from that website (free). Then search ($) Archives for Margaret Heckstall.
Obtain the GEDCOM file in new tab from that website (free). Then search ($) Archives for Sarah Kevington.


text:
Le prêtre canonique don Giovanni Maria Dezan (aussi dit Gian Maria ou Giammaria), maître en éloquence, professeur au séminaire et examinateur synodique, est un écrivain italien du .

Biographie

Tel son contemporain Giannantonio Moschini et , Giovanni Maria Dezan consacra une bonne partie de son œuvre à la ville de Venise. Il a publié des œuvres sur l'histoire de la République de Venise, son clergé, ses églises et ses saints ainsi que des recherches toponymiques sur les rues, ponts et canaux de la ville.

Bibliographie
 Santi Viniziani
 Biografia dei preti Veneziani illustri per letteratura, Venezia, 1835
 Quaranta immagini de' santi e beati viniziani piu noti, Venezia, D. Contarini, 1832

Clergé italien du XIXe siècle
Prêtre catholique italien
Historien italien du XIXe siècle
Histoire de Venise
Naissance en 1771
Naissance à Venise
Décès en janvier 1841
Décès à Venise

redpajama_set_name: RedPajamaWikipedia


text:
Irresponsible M'sian Doctor Makes Patient Wait 4 Hours For Treatment Despite Coughing Up Blood
Husna Halim
Source: Twitter & Gaya Hidup
We've all experienced governmental hospitals at some time or the other before. The waiting times are long and the services provided can at times be rough. It's not all bad, but really good ones only come once in a blue moon.
At times, they might be too busy to entertain all of their patients but in this story, we believe that each of the officers involved could've done more.
According to a post on Twitter written by @sayaUMIE two days ago (Oct 6th), she narrated her friend's life and death complication caused by an ignorant doctor and a few too-easygoing officers.
Hafizah, the patient, started to cough out thick blood at 5 am and requested for her housemate, Umie, to call an ambulance for her as she insisted that the hospital would not entertain them as well if they were to submit normally like other patients. Ambulances will give the impression of emergencies, you see.
When the ambulance arrived, the officers kept on asking questions of the suffocating patient despite seeing her in pools of blood. Can't you at least help her breathe first?!
At Hospital Sultan Ismail, Johor Bahru, Hafizah was sent to the yellow zone and later was made to take a number and WAIT with the other patients because her cough toned down.
So they waited for 2 hours, just to be sent out again because the doctor insisted that she should have come according to the time in the appointment given to her before.
Even when the two explained that Hafizah was coughing bags of blood, the doctor shooed them away by saying that she couldn't do anything outside of the appointed time and asked them to wait again for blood test results, which meant another two hours of waiting. That's 4 hours in total! It was taking so long that Hafizah's parents had arrived from Tangkak.
It was when the patient had another suffocating attack and chest pain that Umie resolved to rush into the doctor's room and ask for help.
Nevertheless, the doctor remarked so casually that it irked Umie, causing her to snap.
"Doctor! My friend is dying out there, you could have at least done something! Like letting her lie down or giving her some kind of medicine. Her cough is as bad as this morning and I can't check her because I'm not the doctor here!"
Finally starting to show some concern, the doctor asked Umie to bring in the patient, who was holding a bag of coughed-up blood – enough to fill up 3 cups.
Source: Gaya Hidup
The doctor saw this and said, "You shouldn't force yourself to cough."
When the patient replied that she was suffocating and that she couldn't breathe, the doctor sassed, "Then you should finish your cough first before trying to breathe."
Hafizah was sent to the yellow zone and when the nurses saw her blood, they took their sweet time to wear plastic aprons and gloves, saying that bloodstains would be hard to wash.
Eventually, a decent doctor checked on Hafizah and yelled in urgency, "Push her to the red zone! Give her oxygen! Give her blood as well!" and called the first doctor out.
After she received an earful from the second doctor, the first doctor called out to Umie and Hafizah's parents while looking all guilty. They were told that Hafizah's health was declining. Her lungs were clogged with blood and that was causing her to suffocate.
The doctor also added that Hafizah couldn't breathe on her own for the time being, so she needed to rely on a machine to breathe. She even had two bags of blood inserted into her body because she had lost so much and was sent to the ICU.
Source: NIH
The first doctor said that she didn't take any action at first because Hafizah looked normal. According to Umie, Hafizah was last seen unconscious with blood-stained clothing. She got out of the ICU for a while but went back in yesterday.
Is this really the standard of doctors in our hospitals? Let's hope that the doctor will truly learn responsibility for her patients and pray that Hafizah will recover soon.
Also read: Baby Shockingly Gets Period Just Five Days After Birth But Doctors Say It's Normal, Here's Why
Related Topics:bad doctorbloodclogged lungcoughcough blooddoctorgovernment doctorsgovernment hospitalgovernment hospitalshospitalsicumalaysian government hospitalsunprofessionalunprofessional doctor


Andre Bradford scores 20 while Derrick Bryant adds 12 points with 14 assists in an 88-56 victory.
Andre Bradford scored 20 points while Derrick Bryant finished with 12 points and 14 assists to lead Harper Woods Chandler Park Academy to an 88-56 boys basketball victory over Macomb Dakota on Saturday in the Horatio Williams Foundation Tournament at Warren Fitzgerald High.
Josh Diggs had 14 points for Chandler Park (10-0). Dakota is 9-3.
River Rouge 66, Flint Beecher 61: Nigel Colvin scored 17 and Donovan Freeman and Micah Parrish 12 each for River Rouge (10-1). Jalen Terry scored 21 for Beecher (9-1).
Warren De La Salle 64, Detroit Edison Academy 56: Michael Sherman scored 22 while Joe Gjonaj, Tyrelle Stephens and Matt Osterhout each had 11 for De La Salle (6-5). Edison Academy is 7-4.
Benton Harbor 90, Ferndale 83: At the Floyd Mayweather Jr. tournament in Grand Rapids, Carlos Johnson scored 34 with 18 rebounds for Benton Harbor (13-0). Sudi McElroy had 35 points and 14 rebounds and Teonta McKeithern scored 23 for Ferndale (6-6).
Detroit Cass Tech 54, Arbor Prep 44: At the Arbor Prep MLK Tournament, Ariah Powell scored 16 while Precious Fields added 14 points and 13 rebounds for Cass Tech (10-2). Arbor Prep is 9-3.


OKD takes advantage of preexisting source control management (SCM) systems hosted either internally (such as an in-house Git server) or externally (for example, on GitHub, Bitbucket, etc.). Currently, OKD only supports Git solutions.
Creating a BuildConfig using a repository, which allows building your application inside of OKD. You can create a BuildConfig manually or let OKD create it automatically by inspecting your repository.


The key is to put yourself first on the list. If you do not take care of yourself then you cannot take care of anyone else. Keep your list short, sweet and something that you can follow and achieve in 2012. Happy Healthy New Year!
Do you ever wonder if the new modern technologies are rotting your brain? Are you dependent on your car or cellphone GPS to get you where you want to go? You may want to re-think that.
Scientists have found that when you use your brain to navigate your route in your car or on foot that you use a map in your head or you find your way on autopilot. Either way you are using the hippocampus area of your brain. If you depend on the GPS you are not.
When scientists have examined the brains of taxi cab drivers in London where they are always looking for shortcuts, they found that their hippocampus was larger than those of non-taxi cab drivers. Older adults who do not use GPS and use maps to navigate have more grey matter than those who don't.
It is possible by activating this part of the brain that we may be able to prevent dementia. This is one instance where it might make sense to do things the old fashioned way. If you need to find a new route to work or go to a place you have not been to in awhile, use a map and navigate on your own.
Another thing that can help is to start calling people the old-fashioned way. Remember their numbers and call them. Do not rely on your smart phone for directions or phone numbers. It can make you stupid!


text:
KHPG in social networks
Donate KHPG
Information Portal «Human Rights in Ukraine»
PROJECTS of KHPG
in English Virtual museum
"Dissident movement in Ukraine" Online library of KHPG
English editions The right
to privacy Stories of convicts
"The struggle for life"
УКР ENG РУС A-z
About KhPG
Political persecution in modern Ukraine
Сampaigns
Our reports to international bodies
Monthly bulletin Prava Ludyny (Human rights)
Some publications KhPG
Freedom of Expression in Ukraine
Аrticles en Français
Politics and human rights
Implementation of European Law
The right to life
Against torture and ill-treatment
The right to liberty and security
The right to a fair trial
Freedom of conscience and religion
Prohibition of discrimination
The right to health care
On refugees
Interethnic relations
Self-government
Civic society
NGO activities
Victims of political repression
Dissidents and their time
Deported peoples
News from the CIS countries
The troubled times
Human Rights Violations associated with EuroMaidan
Human Rights Abuses in Russian-occupied Crimea
The constitution and human rights
The right to privacy
Prevention of discrimination and inequality
The Security service in a constitutional democracy
Analysis of the human rights situation in Ukraine
Human rights in Ukraine – 2004. Human rights organizations' report
Human rights in Ukraine 2009 – 2010. Human rights organizations' report
Human rights in Ukraine 2011. Human rights organizations' report
Human rights and civil society
History of the dissident movement in Ukraine
Voices of war
Documenting war crimes in Ukraine.
The Tribunal for Putin (T4P) global initiative was set up in response to the all-out war launched by Russia against Ukraine in February 2022.
Stalin, Putin and 'annexation of Crimea' adorn new Russian Defence Ministry churchIn Russia level of support for murderous dictator Stalin reaches record highDetained in Putin's Russia for shouting that Stalin should burn in hellTelling the truth about Stalin and the Great Terror gets historians arrested in RussiaRussia arrests second historian of Stalin's TerrorRussia turns to politically-motivated excavations to rewrite history of Soviet Terror after jailing one of its main historiansRussia brings new fabricated charges against historian of the Terror Yuri DmitrievRussia reinstates political trial of historian of the Terror Yuri DmitrievRussian Historian of the Terror Yuri Dmitriev acquitted of main chargesRussian historian faces 9-year sentence for exposing the victims and perpetrators of Stalin's TerrorSerbsky Institute dismisses insane charges in Russia's trial of historian of the Terror Yury DmitrievRussia turns to punitive psychiatry as its trial of jailed historian of the Terror collapsesThe KHPG receives half of Memorial's Nobel, and will give the money to the victims of Russia's invasion of UkraineMoscow tries to justify its war crimes in Ukraine by glorifying its soldiers as 'heroes of Russia' Children trained to be 'Putin's faithful soldiers' in Russian-occupied Crimea No end to Russia's brutal reprisals against imprisoned 66-year-old historian of the Soviet Terror Ukrainian kids from occupied Melitopol indoctrinated and enlisted in Russia's 'Youth Army' UK law firm launches legal action against Russia's Wagner mercenaries over terrorism and war crimes in Ukraine'Mogilisation'. Anti-war activities in Russia, 26 September to 1 OctoberFrom a Zwastika to "No to the War". Anti-war activities in Russia, 9-18 September
• Topics / News from the CIS countries
• Topics / Human Rights Abuses in Russian-occupied Crimea
Russian Deputy Defence Minister: 'Why should we be ashamed of Stalin?'
Halya Coynash
While Russian President Vladimir Putin's face has – for now – been removed from a mosaic in the new Russian Defence Ministry Cathedral, mass murderer Joseph Stalin remains. This is our history, the Deputy Minister of Defence claims, and what is there to be ashamed of?
If you listen to Andrei Kartapolov and watch only Russian state-controlled media, then this would all seem quite normal. In Kartapolov's words, this was a man who "took upon himself all the burden of the war, made the most important decisions. Yes, and in general reinstated religion. Why should we be ashamed of him? Because some people from abroad tell us to? We will decide ourselves who to honour, who to portray on frescos. They remove monuments to the victors because they are afraid of historical truth, they are afraid of dead Soviet marshals and soldiers. At the same time, they understand how disgustingly they are behaving and how great the feat of the fighters and commanders who perished so that they could now live. They are tormented by shame for their inadequacy because they are fighting memorials. We will not do this. We are a great nation and we have the right to our heroes".
The Deputy Defence Minister's interview was published two days before the Russian prosecutor in Tver ordered the dismantling of memorial plaques honouring the victims of Stalin's Terror and of the Katyń Massacre, when the Soviet NKVD, on the orders of Stalin's deputy, executed 22 thousand Polish officers and others, taken prisoner after the USSR invaded Poland on 17 September 1939 (details here). Those memorials, like the mass graves of victims of the Terror at Sandarmokh and the Gulag museum at Perm-36, are destroyed or the facts about them distorted in Putin's Russia with impunity. It is the memorials, erected by force, in the countries that the Soviet Union liberated from the Nazis, only to subjugate, that Russia is now defending.
As the current regime is defending Stalin, a sadist, mass murderer and invader, who collaborated with Adolf Hitler for close on the first two years of World War II. Had it not been for crucial material, petrol, etc. that the USSR was supplying to Nazi Germany right up to the latter's invasion of Soviet Ukraine on 22 June 1941, Hitler might have found it harder to overrun so much of Europe. Had it not been for Stalin's insane, as well as evil, purges of the military on the eve of the War, and even after the Nazi invasion, far less Soviet soldiers and civilians would have perished in the War.
These are facts, however few Russian these days have much chance of learning them. It is the myth around the Soviet war effort under the wise command of Stalin that Russians learn about.
Nor is it particularly safe to try to establish the truth. Reposting a picture in which Putin is likened to Hitler because "both began with the Olympics"; "both invaded Ukraine" and "both shamed their countries before the entire world" can get you jailed for three days in Putin's Russia, while reposting the correct statement that the USSR invaded Poland in 1939 can lead to a huge fine. Not to mention the fact that Yuri Dmitriev, historian of the Terror, has been in prison for three and a half years, as was Sergei Koltyrin, another historian, researching Stalin's crimes, until his death on 2 April 2020.
Systematic 'rehabilitation' of the bloody dictator began back in 2007. After a speech given by Putin to teachers in June 2007, a manual by Alexander Filippov appeared, describing Stalin as "one of the most successful leaders of the USSR". Stalin's aim in both domestic and foreign policy, it was stated, had been the restoration of the Russian Empire. The purges had "led to the formation of a new governing class, able to cope with the task of modernization given the shortage of resources – unwaveringly loyal to the upper echelons of power and irreproachable from the point of view of executive discipline". It was material like this that by October that year made Putin welcome "certain positive moves", noting that "up till quite recently we read things in textbooks that made our hair stand on end…"
In July 2015, as a new Stalin museum and monument were unveiled in Tver, an article was published by Russia's Culture Minister Vladimir Medinsky in which he effectively defended Stalin's role in the country's history. In August 2016, Putin appointed Olga Vasilyeva, an apologist for Stalin, as Education Minister.
Now it seems clear that other apologists head the Russian Defence Ministry, since Katapolov would not be speaking like this if he did not have the support of his superiors. It is immensely disturbing that all of this monstrous lie has the support of the hierarchy of the Russian Orthodox Church, which is tolerating the desecration of a Cathedral with images of Stalin and glorifying Russia's invasion and annexation of Crimea. Although the publicity over the 'Crimea is ours!' mosaic, with Putin, Defence Minister Sergei Shoigu and Russian invaders of Crimea, led to the mosaic being removed from the wall, this is likely only temporary, and another image, celebrating Russia's illegal annexation of Ukrainian territory, is to replace it.
Katapolov's assertion that Stalin 'reinstated religion' is a cynical lie. Yes, in 1941, after understanding that his ally Hitler had "treacherously" betrayed him by invading the USSR, Stalin addressed the Soviet people he had been persecuting and murdering with the words "Brothers and sisters", and he allowed churches to reopen. The churches that his regime had destroyed, while also murdering hundreds of thousands of religious figures, or sending them to labour camps..
While the Deputy Defence Minister claimed that the Cathedral's location (in Moscow region) marks the place where the Nazi advance on Moscow was stopped, the site is also linked with Russia's annexation of Crimea. Kubinka is where the 46th paratroopers brigade which seized control of the Crimean parliament on 27 February 2014 is based.
Although there were always Crimean communists (in particular) who glorified Stalin, it is under Russian occupation that the mass murderer, guilty for the Deportation of the entire Crimean Tatar people, Holodomor and countless heinous crimes, is increasingly seen on posters, etc. Within a year of annexation, a monument to Stalin and the leaders of Great Britain and the USA had been erected in the place where the men signed the Yalta Agreement, and a memorial plaque had been established on a façade in Simferopol. This is while Crimean Tatars have been prohibited from holding remembrance events to honour the victims of the Deportation, a horrific act of genocide.
More about the Defence Ministry's Cathedral and who it honours here Stalin, Putin and 'annexation of Crimea' adorn new Russian Defence Ministry church
NewsMail Subscribe RSS Feed
For reprinting a link to the source is required. The thoughts and reasoning of the authors do not always coincide with the views of the KHPG members
The mode of saving Internet traffic by reducing the size of images
Сontrasting design theme (visually impaired)


July 15, 2013 – The results of the Avenue magazine/Fiasco Gelato "Give us a Taste" contest are in, and it's a tie. The contest, which launched June 3, invited Calgarians to submit unique gelato flavour ideas. The panel of judges was so impressed by all the finalists it was unable to determine a single grand prize winner. The result was a surprise double winner.
The winning flavours of "Give Us a Taste" are Fat Elvis (peanut butter, banana and bacon) by Dani Ossa, and Mimosa with Pop Rocks (champagne, orange, pop rocks and gold flecks) by Ryan Massel (@immrfabulous). As winners, Ossa and Massel will each receive free gelato for a year and the opportunity to work alongside Fiasco Gelato to create their winning flavours for retail. Fat Elvis and Mimosa with Pop Rocks will be sold on the Fiasco Gelato food truck and at the Fiasco Pop-Up Gelateria in Bridgeland Market throughout the month of August.
"The response to the contest was overwhelming. Avenue readers really love their gelato, and their sophisticated and adventurous palates clearly shone through in the 200-plus flavour combinations they came up with," said Jennifer Hamilton, executive editor of Avenue.
Fiasco Gelato selected the six finalists after the contest closed June 30. Among the submissions were flavours as simple as chocolate and as unexpected as chicken nugget. On July 15, Jennifer Hamilton and Fiasco Gelato chief idea officer James Boettcher joined Tara Slone on Breakfast Television Calgary to announce the six unique flavour finalists. In addition to the two winning entries, finalist flavours included Fig and Prosciutto, Nanaimo Bar, Ginger Lemon Crème Brûlée and Pear Prosecco with Rosemary. Each finalist will receive a $25 gift card to Toscana Italian Grill.
Following the announcement of the six finalists on Breakfast Television Calgary, a blue-ribbon panel of culinary experts met at Fiasco Gelato headquarters to select a grand-prize winner. The panel included Jennifer Hamilton (@jenntotheh); James Boettcher (@jamesboettcher); John Gilchrist, food critic and writer (@gilchristjohn); Connie DeSousa and John Jackson, co-chefs and co-owners of Charcut Roast House in Calgary (@charcut); Julie Van Rosendaal, food writer and critic (@dinnerwithjulie); Dan Clapson, food writer (@dansgoodside); and Tara Slone, host of Breakfast Television Calgary (@taratheslone). It took the panel much deliberation and several tasting sessions to arrive at its decision.
"Give Us a Taste" winners Fat Elvis and Mimosa with Pop Rocks will be featured flavours on the Fiasco Gelato food truck at Avenue's first ever Best Neighbourhoods Block Party on August 17.
Fiasco Gelato (@FiascoGelato) is a Calgary artisan gelateria that's trying to make the world a better place, one tiny spoonful at a time. Established in 2003, Fiasco creates premium gelato the old-fashioned way: in small batches, simply, honestly and filled with passion.
Avenue (@AvenueMagazine) is Calgary's premier city and lifestyle magazine. Since 1994, Avenue has delivered award-winning design and intelligent, provocative perspectives to an audience of active, affluent and sophisticated Calgarians. Avenue tells Calgary's stories monthly through architecture, personalities, arts, fashion, food, events and urban life.


text:
\section{Introduction}
We consider the following unconstrained optimization problem, which appears frequently in machine learning and statistics, such as empirical risk minimization:
\begin{equation}\label{prob_def}
	\min_{x\in \mathbb{R}^d}{f(x) \triangleq \frac{1}{n} \sum_{i=1}^n{f_i(x)}},
\end{equation}
where each $f_i$ is $L$-smooth and convex, $f$ is $\mu$-strongly convex.\footnote{In fact, we will only use a weaker quadratic growth assumption. The formal definitions are in Section \ref{sec:preliminaries}. } We further assume that the computation of each $f_i$ is \textit{sparse}, i.e., the computation is supported on a set of coordinates $T_i \subseteq \{1,\ldots, d\}$. For example, generalized linear models (GLMs) satisfy this assumption. GLMs has the form: $\forall i, f_i(x) = \ell_i(\innr{a_i, x})$, where $\ell_i$ is some loss function and $a_i\in \mathbb{R}^d$ is a data sample. In this case, the support of each $f_i$ is the set of non-zero coordinates of $a_i$. Many large real-world datasets have very high sparsity (see Table \ref{data-table} for several examples).

In the past decade, exciting progress has been made on solving problem \eqref{prob_def} with the introduction of stochastic variance reduced methods, such as SAG \citep{SAG,SAG2}, SVRG \citep{SVRG,Prox_SVRG}, SAGA \citep{SAGA}, S2GD \citep{S2GD}, SARAH \citep{SARAH}, etc. These methods achieve fast linear convergence rate as gradient descent (GD), while having low iteration cost as stochastic gradient descent (SGD). There are also several recent works on further improving the practical performance of these methods based on past gradient information \citep{HofmannLLM15,TanMDQ16,ShangLZCNY18,shi2021ai,dubois2021svrg}. Inspired by Nesterov's seminal works \citep{AGD1,AGD2,AGD3}, stochastic accelerated methods have been proposed these years \citep{allen2017katyusha,MiG,SSNM,pmlr-v124-zhou20a,VARAG,VRADA,Anita}. These methods achieve the optimal convergence rates that match the lower complexity bounds established in \citep{woodworthS16}. For sparse problems, lagged update \citep{SAG2} is a common technique for the above methods to efficiently leverage the sparsity.

Inspired by the emerging parallel computing architectures such as multi-core computer and distributed system, many parallel variants of the aforementioned methods have been proposed, and there is a vast literature on those attempts.
Among them, asynchronous lock-free algorithms are of special interest.
\citet{Hogwild} proposed an asynchronous lock-free variant of SGD called Hogwild!\ and first proved that it can achieve a \textit{linear speed-up}, i.e., running Hogwild!\ with $\tau$ parallel processes only requires $O(1 / \tau)$-times the running time of the serial variant. The condition on the maximum overlaps among the parallel processes is called the \textit{linear speed-up condition}. Following \citet{Hogwild}, \citet{SaZORR15,LianHLL15} analyzed asynchronous SGD for non-convex problems, and \citet{DuchiJM13} analyzed for stochastic optimization; \citet{ManiaPPRRJ17} refined the analysis framework (called the perturbed iterate framework) and proposed KroMagnon (asynchronous SVRG); \citet{ASAGA} simplified the perturbed iterate analysis and proposed ASAGA (asynchronous SAGA), and in the journal version of this work,  \citet{ASAGA_JMLR} further improved the analysis of KroMagnon and Hogwild!; \citet{ProxASAGA} derived proximal variant of ASAGA; \citet{NguyenNDRST18} refined the analysis of Hogwild!; \citet{JoulaniGS19} proposed asynchronous methods in online and stochastic settings; \citet{gu2020unified} proposed several asynchronous variance reduced algorithms for non-smooth or non-convex problems; \citet{StichMJ21} studied a broad variety of SGD variants and improved the speed-up condition of asynchronous SGD.

All the above mentioned asynchronous methods are based on non-accelerated schemes such as SGD, SVRG and SAGA. Witnessing the success of recently proposed stochastic accelerated methods, it is natural to ask: \textit{Is it possible to achieve stochastic acceleration in asynchronous lock-free optimization? Will it lead to a worse linear speed-up condition?} The second question comes from a common perspective that accelerated methods are less tolerant to the gradient noise, e.g., \citep{devolder2014first,RMM}.

In this work, we give positive answers to these questions, i.e., we propose the first asynchronous stochastic accelerated method for solving problem~\eqref{prob_def}, and we prove that it requires the identical linear speed-up condition as the non-accelerated counterparts.

The works that are closest related to us are \citep{fang2018accelerating,MiG,A2BCD}.  \citet{fang2018accelerating} proposed several asynchronous accelerated schemes. However, their analysis requires consistent read and does not consider sparsity. The dependence on the number of overlaps $\tau$ in their complexities is $O(\tau)$ compared with $O(\sqrt{\tau})$ in our result. \citet{MiG} naively extended their proposed accelerated method into asynchronous lock-free setting. However, even in the serial case, their result imposes strong assumptions on the sparsity. This issue is discussed in detail in Section \ref{sec:svc}. In the asynchronous case, no theoretical acceleration is achieved in \citep{MiG}. \citet{A2BCD} proposed the first asynchronous accelerated block-coordinate descent method (BCD). However, as pointed out in Appendix F in \citep{ProxASAGA}, BCD methods perform poorly in sparse problems since they  focus only on a single coordinate of the gradient information.

\section{Preliminaries}
\label{sec:preliminaries}
\paragraph{Notations} We use $\innr{\cdot,\cdot}$ and $\norm{\cdot}$ to denote the standard inner product and the Euclidean norm, respectively. We let $[n]$ denote the set $\{1,2,\ldots,n\}$, $\mathbb{E}$ denote the total expectation and $\mathbb{E}_{i}$ denote the expectation with respect to a random sample $i$. We use $[x]_v$ to denote the coordinate $v$ of the vector $x\in \mathbb{R}^d$, and for a subset $T \subseteq [d]$, $[x]_T$ denotes the collection of $[x]_v$ for $v \in T$. We let $\lceil \cdot \rceil$ denote the ceiling function, $I_d$ denote the identity matrix and $(\alpha)_+ \triangleq \max{\{\alpha, 0\}}$.

We say that a function $f: \mathbb{R}^d \rightarrow \mathbb{R}$ is \textit{$L$-smooth} if it has $L$-Lipschitz continuous gradients, i.e., $\forall x, y\in \mathbb{R}^d$,
\[
\norm{\pf{x} - \pf{y}} \leq L\norm{x - y}.
\]
An important consequence of $f$ being $L$-smooth and convex is that $\forall x, y\in \mathbb{R}^d$,
\[
f(x) - f(y) - \innr{\pf{y}, x - y}  \geq{} \frac{1}{2L} \norm{\pf{x} - \pf{y}}^2.
\]
We call it the \textit{interpolation condition} following \citep{taylor2017smooth}. A continuously differentiable $f$ is called \textit{$\mu$-strongly convex} if $\forall x, y\in \mathbb{R}^d$,
\[
f(x) - f(y) - \innr{\pf{y}, x - y} \geq \frac{\mu}{2} \norm{x - y}^2.
\]
In particular, the strong convexity at any $x\in \mathbb{R}^d$ and $x^{\star}$ is called the \textit{quadratic growth}, i.e.,
\[
f(x) - f(x^{\star}) \geq \frac{\mu}{2} \norm{x - x^{\star}}^2.
\]
All our results hold under this weaker condition. We define $\kappa \triangleq \frac{L}{\mu}$, which is always called the condition ratio. Other equivalent definitions of these assumptions can be found in \citep{AGD3, karimi2016linear}.

Oracle complexity refers to the number of incremental gradient computations needed to find an $\epsilon$-accurate solution (in expectation), i.e., $\E{f(x)} - f(x^{\star}) \leq \epsilon$.

\section{Serial Sparse Accelerated SVRG}

We first introduce a new accelerated SVRG variant with sparse updates in the serial case (Algorithm~\ref{alg:SS-Acc-SVRG}), which serves as the base algorithm for our asynchronous scheme. Our technique is built upon the following sparse approximated SVRG gradient estimator proposed in \citep{ManiaPPRRJ17}: For uniformly random $i\in [n]$ and $y, \tilde{x} \in \mathbb{R}^d$, define
\begin{equation}\label{S-SVRG-estimator}
	\mathcal{G}_y \triangleq \pfi{y} - \pfi{\tilde{x}} + D_i\pf{\tilde{x}},
\end{equation}
where $D_i \triangleq P_i D$ with $P_i\in \mathbb{R}^{d\times d}$ being the diagonal projection matrix of $T_i$ (the support of $f_i$) and $D = (\frac{1}{n}\sum_{i=1}^n{P_i})^{-1}$, which can be computed and stored in the first pass. Note that we assume the coordinates with zero cardinality have been removed in the objective function,\footnote{Clearly, the objective value is not supported on those coordinates.} and thus $nI_d \succeq D \succeq I_d$. A diagonal element of $D$ corresponds to the inverse probability of the coordinate belonging to a uniformly sampled support $T_i$. It is easy to verify that this construction ensures the unbiasedness $\Er{i}{D_i\pf{\tilde{x}}} = \pf{\tilde{x}}$.

This section is organized as follows: In Section \ref{sec:svc}, we summarize the key technical novelty and present the convergence result; in Section \ref{sec:remarks}, we remark on the design of Algorithm \ref{alg:SS-Acc-SVRG}; in Section \ref{sec:general_convex}, we briefly discuss its general convex variant.

\subsection{Sparse Variance Correction}
\label{sec:svc}

\begin{algorithm*}[t]
	\caption{SS-Acc-SVRG: Serial Sparse Accelerated SVRG}
	\label{alg:SS-Acc-SVRG}
	\renewcommand{\algorithmicrequire}{\textbf{Input:}}
	\renewcommand{\algorithmicensure}{\textbf{Initialize:}}
	\begin{algorithmic}[1]
		\Require initial guess $x_0 \in \mathbb{R}^d$, constant $\omega > 1$ which controls the restart frequency.
		\Ensure set the scalars $m, \vartheta, \varphi, \eta,  S, R$ according to Theorem \ref{thm:SS-Acc-SVRG}, initialize the diagonal matrix $D$.
		\For{$r=0, \ldots, R-1$}\Comment{performing restarts}
		\State $\tilde{x}_0 = z^0_0 = x_r$.
		\For{$s=0, \ldots, S-1$}
		\State Compute and store $\pf{\tilde{x}_s}$.
		\For{$k = 0, \ldots, m-1$}
		\State Sample $i_k$ uniformly in $[n]$ and let $T_{i_k}$ be the support of $f_{i_k}$.
		\State\label{alg-step:coupling} $\supp{y_{k}} = \vartheta\cdot \supp{z^s_k} + \left(1 - \vartheta\right) \cdot\supp{\tilde{x}_s} - \varphi\cdot\supp{D\pf{\tilde{x}_s}}.$ \Comment{sparse coupling}
		\State\label{alg-step:mirror_c} $\supp{z^s_{k+1}} = \supp{z^s_k} - \eta \cdot\big(\pfik{\supp{y_k}} - \pfik{\supp{\tilde{x}_s}} + D_{i_k}\pf{\tilde{x}_s}\big)$.\Comment{sparse update}
		\EndFor
		\State $\tilde{x}_{s+1} = y_t$ for uniformly random $t\in \{0, 1, \ldots, m-1\}$.
		\State $z^{s+1}_0 = z^s_m$.
		\EndFor
		\State $x_{r+1} = \frac{1}{S}\sum_{s = 0}^{S-1}{\tilde{x}_{s+1}}$.
		\EndFor
		\renewcommand{\algorithmicensure}{\textbf{Output:}}
		\Ensure $x_R$.
	\end{algorithmic}
\end{algorithm*}

Intuitively, the sparse SVRG estimator \eqref{S-SVRG-estimator} will leads to a larger variance compared with the dense one (when $D=I_d$). In the previous attempt, \citet{MiG} naively extends an accelerated SVRG variant into the sparse setting, which results in a fairly strong restriction on the sparsity as admitted by the authors.\footnote{It can be shown that when $\kappa$ is large, almost no sparsity is allowed in their result.} By contrast, sparse variants of SVRG and SAGA in \citep{ManiaPPRRJ17,ASAGA} require no assumption on the sparsity, and they achieve the same oracle complexities as their original dense versions.

Analytically speaking, the only difference of adopting the sparse estimator \eqref{S-SVRG-estimator} is on the variance bound. The analysis of non-accelerated dense SVRG typically uses the following variance bound ($D=I_d$) \citep{SVRG, Prox_SVRG}:
\[
\E{\norm{\mathcal{G}_y - \pf{y}}^2} \leq 4L\big(f(y) - f(x^{\star}) + f(\tilde{x}) - f(x^{\star})\big).
\]
It is shown in \citep{ManiaPPRRJ17} that the sparse estimator \eqref{S-SVRG-estimator} admits the same variance bound for any $D$. Thus, the analysis in the dense case can be directly applied to the sparse variant, which leads to a convergence guarantee that is independent of the sparsity.

However, things are not as smooth in the accelerated case. To (directly) accelerate SVRG, we typically uses a much tighter variance bound in the form ($D=I_d$) \citep{allen2017katyusha}:
\[
\E{\norm{\mathcal{G}_y - \pf{y}}^2} \!\leq 2L\big(f(\tilde{x}) - f(y) - \innr{\pf{y}, \tilde{x} - y}\big).
\]
Unfortunately, in the sparse case ($D\neq I_d$), we do not have an identical variance bound as before. The variance of the sparse estimator \eqref{S-SVRG-estimator} can be bounded as follows. The proof is given in Appendix \ref{app:lemma_vb}.
\begin{lemmas}[Variance bound for accelerated SVRG]
	\label{lem:S-SVRG-VB}
	The variance of $\mathcal{G}_y$ (defined at \eqref{S-SVRG-estimator}) can be bounded as
	\begin{equation}\label{S-SVRG-VB}
		\begin{aligned}
			\Ei{\norm{\mathcal{G}_y - \pf{y}}^2}
			\leq{}& 2L\big(f(\tilde{x}) - f(y) - \innr{\pf{y}, \tilde{x} - y}\big)  \!-\! \norm{\pf{y}}^2 \\
			&+ \underbrace{2\innr{\pf{y}, D \pf{\tilde{x}}}}_{\mathcal{R}_1} -\innr{\pf{\tilde{x}},D\pf{\tilde{x}}}.
		\end{aligned}
	\end{equation}
\end{lemmas}
In general, except for the dense case, where we can drop the last three terms above by completing the square, this upper bound will always be correlated with the sparsity (i.e., $D$). This correlation causes the strong sparsity assumption in \citep{MiG}. We may consider a more specific case where $D = nI_d$. In this case, the last three terms above can be written as $(n-1)\norm{\pf{y}}^2 - n\norm{\pf{y} - \pf{\tilde{x}}}^2$, which is not always non-positive.

Inspecting \eqref{S-SVRG-VB}, we see that it is basically the term $\mathcal{R}_1$, which could be positive, that causes the issue. We thus propose a novel \textit{sparse variance correction} for accelerated SVRG, which is designed to perfectly cancel $\mathcal{R}_1$. The correction is added to the coupling step (Step \ref{alg-step:coupling}):
\[
y_{k} = \vartheta \cdot z_k + \underbrace{\left(1 - \vartheta\right)\cdot \tilde{x}_s}_{\text{Negative Momentum}} - \underbrace{\varphi \cdot D\pf{\tilde{x}_s}}_{\text{Sparse Variance Correction}}.
\]
This correction neutralizes all the negative effect of the sparsity, in a similar way as how the negative momentum cancels the term $\innr{\pf{y}, \tilde{x} - y}$ in the analysis \citep{allen2017katyusha}. We can understand this correction as a variance reducer that controls the additional sparse variance. Then we have the following sparsity-independent convergence result for Algorithm~\ref{alg:SS-Acc-SVRG}, and its proof is given in Appendix \ref{app:thm_serial}.

\begin{theorems}
	\label{thm:SS-Acc-SVRG}
	For any constant $\omega > 1$, let $m=\Theta(n), \vartheta=\frac{\sqrt{m}}{\sqrt{\kappa} + \sqrt{m}}, \varphi = \frac{1 - \vartheta}{L}$, $\eta = \frac{1 - \vartheta}{L\vartheta}$ and $S= \left\lceil 2\omega\sqrt{\frac{\kappa}{m}}\right\rceil$. For any accuracy \mbox{$\epsilon > 0$,} we restart $R= O\big(\log{\frac{f(x_0) - f(x^{\star})}{\epsilon}}\big)$ rounds. Then, Algorithm \ref{alg:SS-Acc-SVRG} outputs $x_R$ satisfying $\E{f(x_R)} - f(x^{\star}) \leq \epsilon$ in oracle complexity
	\[
	O\left(\max{\left\{n, \sqrt{\kappa n}\right\}}\log{\frac{f(x_0) - f(x^{\star})}{\epsilon}}\right).
	\]
\end{theorems}

This complexity matches the lower bound established in \citep{woodworthS16} (up to a log factor), and is substantially faster than the $O\big((n+\kappa)\log{\frac{1}{\epsilon}}\big)$ complexity of sparse approximated SVRG and SAGA derived in \citep{ASAGA, ASAGA_JMLR} in the ill-conditioned regime ($\kappa \gg n$).

\subsection{Other Remarks about Algorithm \ref{alg:SS-Acc-SVRG}}
\label{sec:remarks}

\begin{itemize}[leftmargin=14pt]
	\item We adopt a restart framework to handle the strong convexity, which is also used in \citep{MiG} and is suggested in \citep{allen2017katyusha} (footnote 9). This framework allows us to relax the strong convexity assumption to quadratic growth. Other techniques for handling the strong convexity such as (i) assuming a strongly convex regularizer and using proximal update \citep{nesterov2005smooth,allen2017katyusha}, and (ii) the direct constructions in \citep{VARAG,zhou2020boosting, Anita} fail to keep the inner iteration sparse, and we are currently not sure how to modify them.

	\item The restarting point $x_{r+1}$ is chosen as the averaged point instead of a uniformly random one because large deviations are observed in the loss curve if a random restarting point is used.

	\item We can also use the correction to fix the sparsity issue in \citep{MiG}, which leads to a slightly different method and somewhat longer proof.

	\item Algorithm \ref{alg:SS-Acc-SVRG} degenerates to a strongly convex variant of Acc-SVRG-G proposed in \citep{zhou2021practical} in the dense case ($D=I_d$), which summarizes our original inspiration. Note that Acc-SVRG-G is designed for minimizing the gradient norm and function value at the same time, which has nothing to do with handling sparsity. Its correction is not necessary if we only want to minimize the function value at the optimal rate, unlike our correction.
\end{itemize}
\subsection{General Convex Variant}
\label{sec:general_convex}
A general convex ($\mu=0$) variant of Algorithm~\ref{alg:SS-Acc-SVRG} can be derived by removing the restarts and adopting a variable parameter choice similar to Acc-SVRG-G in \citep{zhou2021practical}, which leads to a similar rate. We also find that our correction can be used in Varag \citep{VARAG} and ANITA \citep{Anita} in the general convex case. Since the previous works  on asynchronous lock-free optimization mainly focus on the strongly convex case, we omit the discussion here.

\begin{algorithm*}[t]
	\caption{AS-Acc-SVRG: Asynchronous Sparse Accelerated SVRG}
	\label{alg:AS-Acc-SVRG}
	\renewcommand{\algorithmicrequire}{\textbf{Input:}}
	\renewcommand{\algorithmicensure}{\textbf{Initialize:}}
	\begin{algorithmic}[1]
		\Require initial guess $x_0 \in \mathbb{R}^d$, constant $\omega > 1$ which controls the restart frequency.
		\Ensure set $m, \vartheta, \varphi, \eta,  S, R$ according to Theorem \ref{thm:AS-Acc-SVRG}, initialize shared variable $z$ and diagonal matrix $D$.
		\For{$r=0, \ldots, R-1$}\Comment{performing restarts}
		\State $\tilde{x}_0 = z = x_r$.
		\For{$s=0, \ldots, S-1$}
		\State Compute in parallel and store $\pf{\tilde{x}_s}$.
		\While{number of samples $\leq m$} \textbf{in parallel}
		\State Sample $i$ uniformly in $[n]$ and let $T_i$ be the support of $f_i$.
		\State $\suppi{\hat{z}}=$ inconsistent read of $z$ on $T_i$.
		\State $\suppi{\hat{y}} = \vartheta\cdot \suppi{\hat{z}} + \left(1 - \vartheta\right)\cdot \suppi{\tilde{x}_s} - \varphi\cdot \suppi{D\pf{\tilde{x}_s}}.$
		\State $\suppi{u} = - \eta \cdot\big(\pfi{\suppi{\hat{y}}} - \pfi{\suppi{\tilde{x}_s}} + D_i\pf{\tilde{x}_s}\big)$.
		\For{$v\in T_i$}
		\State $[z]_v = [z]_v + [u]_v$. \Comment{coordinate-wise atomic write}
		\EndFor
		\EndWhile
		\State $\tilde{x}_{s+1} = \hat{y}_t$, where $\hat{y}_t$ is chosen uniformly at random among the inconsistent $\hat{y}$ in the previous epoch.
		\EndFor
		\State $x_{r+1} = \frac{1}{S}\sum_{s =0}^{S-1}{\tilde{x}_{s+1}}$.
		\EndFor
		\renewcommand{\algorithmicensure}{\textbf{Output:}}
		\Ensure $x_R$.
	\end{algorithmic}
\end{algorithm*}

\section{Asynchronous Sparse Accelerated SVRG}
\label{sec:async}
We then extend Algorithm \ref{alg:SS-Acc-SVRG} into the asynchronous setting (Algorithm \ref{alg:AS-Acc-SVRG}), and analyze it under the perturbed iterate framework proposed in \citep{ManiaPPRRJ17}. Note that Algorithm \ref{alg:AS-Acc-SVRG} degenerates into Algorithm \ref{alg:SS-Acc-SVRG} if there is only one thread (or worker).

\paragraph{Perturbed iterate analysis} let us denote the $k$th update as $\mathcal{G}_{\hat{y}_k} = \pfik{\hat{y}_k} - \pfik{\tilde{x}_s} + D_{i_k} \pf{\tilde{x}_s}$. The precise ordering of the parallel updates will be defined in the next paragraph. \citet{ManiaPPRRJ17} proposed to analyze the following virtual iterates:
\begin{equation}\label{virtual_iterate}
	z_{k+1} = z_k - \eta\cdot\mathcal{G}_{\hat{y}_k}, \text{ for } k = 0, \ldots, m-1.
\end{equation}
They are called the virtual iterates because that except for $z_0$ and $z_m$, other iterates may not exist in the shared memory due to the lock-free design. Then, the inconsistent read $\hat{z}_k$ is interpreted as a perturbed version of $z_k$, which will be formalized shortly. Note that $z_m$ is precisely $z$ in the shared memory after all the one-epoch updates are completed due to the atomic write requirement, which is critical in our analysis.

\paragraph{Ordering the iterates} An important issue of the analysis in asynchrony is how to correctly order the updates that happen in parallel. \citet{Hogwild} increases the counter $k$ after each successful write of the update to the shared memory, and this ordering has been used in many follow-up works. Note that under this ordering, the iterates $z_k$ at \eqref{virtual_iterate} exist in the shared memory. However, this ordering is incompatible with the unbiasedness assumption (Assumption \ref{assum:indp} below), that is, enforcing the unbiasedness would require some additional overly strong assumption. \citet{ManiaPPRRJ17} addressed this issue by increasing the counter $k$ just before each inconsistent read of $z$. In this case, the unbiasedness can be simply enforced by reading all the coordinates of $z$ (not just those on the support). Although this is expensive and is not used in their implementation, the unbiasedness is enforceable under this ordering, which is thus more reasonable.  \citet{ASAGA_JMLR} further refined and simplified their analysis by proposing to increase $k$ after each inconsistent read of $z$ is completed. This modification removes the dependence of $\hat{z}_k$ on ``future'' updates, i.e., on $i_r$ for $r > k$, which significantly simplifies the analysis and leads to better speed-up conditions. See \citep{ASAGA_JMLR} for more detailed discussion on this issue. We follow the ordering of \citep{ASAGA_JMLR} to analyze Algorithm \ref{alg:AS-Acc-SVRG}. Given this ordering, the value of $\hat{z}_k$ can be explicitly described as
\[
[\hat{z}_k]_v = [z_0]_v - \eta\sum_{\substack{j\in\{0,\ldots, k-1\}\\ \text{s.t. coordinate $v$ was}\\ \text{written for $j$ before $k$}}} {[\mathcal{G}_{\hat{y}_j}]_v}.
\]
Since $\hat{y}$ is basically composing $\hat{z}$ with constant vectors, it can also be ordered as \begin{equation} \label{def_inconsistency_y}\hat{y}_k = \vartheta\hat{z}_k + \left(1 - \vartheta\right)\tilde{x}_s - \varphi D\pf{\tilde{x}_s}.
\end{equation}

\paragraph{Assumptions} The analysis in this line of work crucially relies on the following two assumptions.
\begin{assumptions}[unbiasedness]\label{assum:indp}
	$\hat{z}_k$ is independent of the sample $i_k$. Thus,  we have $\E{\mathcal{G}_{\hat{y}_k} | \hat{z}_k} = \pf{\hat{y}_k}$.
\end{assumptions}
As we mentioned before, the unbiasedness can be enforced by reading all the coordinates of $z$ while in practice one would only read those necessary coordinates. This inconsistency exists in all the follow-up works that use the revised ordering \citep{ManiaPPRRJ17,ASAGA_JMLR, ProxASAGA,MiG,JoulaniGS19, gu2020unified}, and it is currently unknown how to avoid. Another inconsistency in Algorithm \ref{alg:AS-Acc-SVRG} is that in the implementation, when a $\hat{y}_k$ is selected as the next snapshot $\tilde{x}_{s+1}$, all the coordinates of $\hat{z}_k$ are loaded. This makes the choice of $\tilde{x}_{s+1}$ not necessarily uniformly random. KroMagnon (the improved version in \citep{ASAGA_JMLR}) also has this issue. In practice, no noticeable negative impact is observed for these two inconsistencies.
\begin{assumptions}[bounded overlaps]
	\label{assum:overlap}
	There exists a uniform bound $\tau$ on the maximum number of iterations that can overlap together.
\end{assumptions}
This is a common assumption in the analysis with stale gradients. Under this assumption, the explicit effect of asynchrony can be modeled as:
\begin{equation}\label{def_inconsistency}
	\hat{z}_k = z_k + \eta\sum_{j=(k-\tau)_{+}}^{k-1}{J^k_j \mathcal{G}_{\hat{y}_j}},
\end{equation}
where $J^k_j$ is a diagonal matrix with its elements in $\{0, 1\}$. The $1$ elements indicate that $\hat{z}_k$ lacks some ``past" updates on those coordinates.

Defining the same sparsity measure as in \citep{Hogwild}: $\Delta = \frac{1}{n}\cdot\max_{v\in[d]}{|\{i: v\in T_i\}|}$, we are now ready to establish the convergence result of Algorithm \ref{alg:AS-Acc-SVRG}. We first present the following guarantee for any $\tau$ given that some upper estimation of $\tau$ is available. The proof is provided in Appendix \ref{app:thm_async}.

\begin{theorems}
	\label{thm:AS-Acc-SVRG}
	For some $\widetilde{\tau} \geq \tau$ and any constant $\omega > 1$, we choose $m=\Theta(n), \vartheta=\frac{\sqrt{m}}{\sqrt{\kappa(1+2\sqrt{\Delta}\widetilde{\tau})} + \sqrt{m}}, \varphi = \frac{1 - \vartheta}{L}$, $\eta =  \frac{(1 - \vartheta)}{L\vartheta(1 + 2\sqrt{\Delta}\widetilde{\tau})}$ and $S= \left\lceil 2\omega\sqrt{\frac{\kappa}{m}(1+2\sqrt{\Delta}\widetilde{\tau})} \right\rceil$. For any accuracy \mbox{$\epsilon > 0$,} we restart $R= O\big(\log{\frac{f(x_0) - f(x^{\star})}{\epsilon}}\big)$ rounds. In this case, Algorithm \ref{alg:AS-Acc-SVRG}   outputs $x_R$ satisfying $\E{f(x_R)} - f(x^{\star}) \leq \epsilon$ in oracle complexity
	\[
	O\left( \max{\left\{n, \sqrt{\kappa n(1+2\sqrt{\Delta}\widetilde{\tau})} \right\}}\log{\frac{f(x_0) - f(x^{\star})}{\epsilon}}\right).
	\]
\end{theorems}

Based on this theorem, it is direct to identify the region of $\tau$ where a theoretical linear speed-up is achievable. The proof is straightforward and is thus omitted.

\begin{corollarys}[Speed-up condition]
	\label{cor:speed-up}
	In Theorem~\ref{thm:AS-Acc-SVRG}, suppose $\tau \leq O\left(\frac{1}{\sqrt{\Delta}}\max{\left\lbrace\frac{n}{\kappa}, 1\right\rbrace}\right)$. Then, setting $\widetilde{\tau} = O\left(\frac{1}{\sqrt{\Delta}}\max{\left\lbrace\frac{n}{\kappa}, 1\right\rbrace}\right)$, we have $S = O\left(\max{\left\lbrace 1, \sqrt{\frac{\kappa}{n}}\right\rbrace}\right)$, which leads to the total oracle complexity $\text{\#grad} = R\cdot S\cdot(n+2m) = O\big(\max{\left\{n, \sqrt{\kappa n}\right\}}\log{\frac{f(x_0) - f(x^{\star})}{\epsilon}}\big)$.
\end{corollarys}

Note that the construction of Algorithm \ref{alg:AS-Acc-SVRG} naturally enforces that $\tau \leq m$. Hence, the precise linear speed-up condition of AS-Acc-SVRG is that $\tau = O(n)$ and $\tau = O(\frac{1}{\sqrt{\Delta}}\max{\left\lbrace\frac{n}{\kappa}, 1\right\rbrace})$, which is identical to that of ASAGA (cf., Corollary 9 in \citep{ASAGA_JMLR}) and slightly better than that of KroMagnon (cf., Corollary~18 in \citep{ASAGA_JMLR}).

\subsection{Some Insights about the Asynchronous Acceleration}
Let us first consider the serial case (Algorithm \ref{alg:SS-Acc-SVRG}). Observe that in one epoch, the iterate $y_k$ is basically composing $z_k$ with constant vectors, we can equivalently write the update (Step \ref{alg-step:mirror_c}) as $y_{k+1} = y_k - \eta\vartheta \cdot \mathcal{G}_{y_k}$. Thus, the inner loop of Algorithm \ref{alg:SS-Acc-SVRG} is identical to that of (sparse) SVRG. The difference is that at the end of each epoch, when the snapshot $\tilde{x}$ is changed, an offset (or momentum) is added to the iterate. This has been similarly observed for accelerated SVRG in \citep{SSNM}. Note that since the sequence $z$ appears in the potential function, the current formulation of Algorithm \ref{alg:SS-Acc-SVRG} allows a cleaner analysis. Then, in the asynchronous case, the inner loop of Algorithm~\ref{alg:AS-Acc-SVRG} can also be equivalently written as the updates of asynchronous SVRG, and the momentum is added at the end of each epoch. This gives us some insights about the identical speed-up condition in Corollary \ref{cor:speed-up}: Since the asynchronous perturbation only affects the inner loop, the momentum is almost uncorrupted, unlike the cases of noisy gradient oracle. That is, the asynchrony only corrupts the ``non-accelerated part'' of Algorithm~\ref{alg:AS-Acc-SVRG}, which thus leads to the same speed-up condition as the non-accelerated methods.

\bgroup
\def1.5{1.5}
\begin{table*}[t]
	\scriptsize
	\caption{Summary of the datasets. Density is the ratio of non-zero elements.} \label{data-table}
	\begin{center}
		\begin{tabular}{|c|l|c|c|c|c|c|c|}
			\hline \textbf{Scale}  &\textbf{Dataset}  &$\boldsymbol{n}$  &$\boldsymbol{d}$
			&$\boldsymbol{\mu}$
			&\textbf{Density} &$\boldsymbol{\Delta}$  &\textbf{Description} \\
			\hhline{|========|}
			\multirow{4}{*}{Small} &Synthetic &\SI{100000}{} &\SI{100000}{} &$10^{-7}$  &$10^{-5}$ &$10^{-5}																																									$ &Identity data matrix with random labels\\
			&\textsf{KDD2010.small}  &\SI{70000}{} &\SI{29890095}{} &$10^{-7}$ &$10^{-6}$ &$0.15$ &The first \SI{70000}{} data samples of \textsf{KDD2010} \\
			&\textsf{RCV1.train}  &\SI{20242}{} &\SI{47236}{}&$10^{-6}$ &$1.6 \cdot 10^{-3}$ &$0.42$ & \\
			&\textsf{News20} &\SI{19996}{} &\SI{1355191}{} &$10^{-6}$ &$3.4 \cdot 10^{-4}$ &$0.93$ & \\
			\hline \multirow{3}{*}{Large} &\textsf{KDD2010} &\SI{19264097}{} &\SI{29890095}{} &$10^{-10}$ &$10^{-6}$ &$0.16$& \\
			&\textsf{RCV1.full}  &\SI{697641}{} &\SI{47236}{} &$10^{-9}$ &$1.5 \cdot 10^{-3}$ &$0.43$ &Combined test and train sets of \textsf{RCV1} \\
			&\textsf{Avazu-site.train}  &\SI{23567843}{} &\SI{999962}{} &$10^{-10}$ &$1.5 \cdot 10^{-5}$ &$0.96$ & \\
			\hline
		\end{tabular}
	\end{center}
\end{table*}
\egroup

\section{Experiments}
We present numerical results for the proposed scheme on optimizing the $\ell_2$-logistic regression problem:
\begin{equation} \label{exp-obj}
	f(x) = \frac{1}{n} \sum_{i=1}^n {\log{\big(1 + \exp{(-b_i\innr{a_i,x})}\big)}} + \frac{\mu}{2}\norm{x}^2,
\end{equation}
where $a_i \in \mathbb{R}^d$, $b_i \in \lbrace -1, +1\rbrace$, $i\in[n]$ are the data samples and $\mu$ is the regularization parameter. We use the datasets from LIBSVM website \citep{LIBSVM}, including \textsf{KDD2010} \citep{yu2010feature}, \textsf{RCV1} \citep{lewis2004rcv1}, \textsf{News20} \citep{keerthi2005modified}, \textsf{Avazu} \citep{JuanZCL16}. All the datasets are normalized to ensure a precise control on $\kappa$. We focus on the ill-conditioned case where $\kappa \gg n$ (the case where acceleration is effective). The dataset descriptions and the choices of $\mu$ are provided in Table \ref{data-table}. We conduct serial experiments on the small datasets and asynchronous experiments on the large ones. The asynchronous experiments were conducted on a multi-core HPC. Detailed setup can be found in Appendix~\ref{app:exp}.

Before presenting the empirical results, let us discuss two subtleties in the implementation:
\begin{itemize}[leftmargin=14pt]
	\item If each $f_i(x) = \log{\big(1 + \exp{(-b_i\innr{a_i,x})}\big)} + \frac{\mu}{2}\norm{x}^2$, then it is supported on every coordinate due to the $\ell_2$-regularization. Following \citep{ASAGA_JMLR}, we sparsify the gradient of the regularization term as $\mu D_i x$; or equivalently, $f_i(x)=\log{\big(1 + \exp{(-b_i\innr{a_i,x})}\big)} + \frac{\mu}{2}\innr{x, D_ix}$. Clearly, this $f_i$ also sums up to the objective \eqref{exp-obj}. The difference is that the Lipschitz constant of $f_i$ will be larger, i.e., from $0.25 + \mu$ to at most $0.25 + \mu n$. Since we focus on the ill-conditioned case ($L \gg \mu n$), this modification will not have significant effect.

	\item In Theorems \ref{thm:SS-Acc-SVRG} and \ref{thm:AS-Acc-SVRG}, all the parameters have been chosen optimally in our analysis except for $\omega$, which controls the restart frequency. Despite all the theoretical benefits of the restart framework mentioned in Section \ref{sec:remarks}, in practice, we do not find performing restarts leads to a faster convergence. Thus, we do not perform restarts in our experiments (or equivalently, we choose a relatively large $\omega = 50$). Clearly, this choice will not make our method a ``heuristic'' since the theorems hold for any $\omega > 1$. Detailed discussion is given in Appendix \ref{app:restart}.
\end{itemize}

Additional experiments for verifying the $\sqrt{\kappa}$ dependence and a sanity check for our implementation is included in Appendix \ref{app:justify} and \ref{app:sanity}, respectively.

\subsection{The Effectiveness of Sparse Variance Correction}

We first study the practical effect of the correction in the serial case. In Figure \ref{fig:SVC}, ``with SVC'' is Algorithm~\ref{alg:SS-Acc-SVRG} and ``without SVC'' is a naive extension of the dense version of Algorithm~\ref{alg:SS-Acc-SVRG} into the sparse case (i.e., simply using a sparse estimator \eqref{S-SVRG-estimator}), which suffers from the same sparsity issue as in \citep{MiG}. For the two variants, we chose the same parameters specified in Theorem \ref{alg:SS-Acc-SVRG} to conduct an ablation study. In theory, we would expect the sparsity issue to be more severe if $D$ is closer to $n I_d$ (i.e., more sparse). Note that by definition, $\Delta = \max_{i\in[d]} {D_{ii}^{-1}}$, and thus $D \succeq \Delta^{-1} I_d$. Hence, smaller $\Delta$ indicates that $D$ is closer to $n I_d$. From Figure \ref{fig:SVC_news20} to \ref{fig:SVC_synthetic}, $\Delta$ is decreasing and we observe more improvement from the correction. The improvement is consistent in our experiments, which justifies the effectiveness of sparse variance correction.

\subsection{Sparse Estimator v.s. Lagged Update}
We then examine the running time improvement from adopting the sparse estimator compared with using the lagged update technique. Lagged update technique handles sparsity by maintaining a last seen iteration for each coordinate. When a coordinate is involved in the current iteration, it computes an accumulated update from the last seen iteration in closed form. Such computation is dropped when adopting a sparse estimator, which is the source of running time improvement. Moreover, it is extremely difficult to extend the lagged update technique into the asynchronous setting as discussed in Appendix E in \citep{ASAGA_JMLR}. In Figure \ref{fig:LU}, we compare SS-Acc-SVRG (Algorithm \ref{alg:SS-Acc-SVRG}) with lagged update implementations of (dense) SS-Acc-SVRG and Katyusha. Their default parameters were used. Note that the lagged update technique is much trickier to implement, especially for Katyusha. We need to hand compute the closed-form solution of some complicated constant recursive sequence for the accumulated update. Plots with respect to effective passes are provided in Appendix \ref{app:justify}, in which SS-Acc-SVRG and Katyusha show similar performance.

\begin{figure*}[t]
	\begin{center}
		\begin{subfigure}{0.245\linewidth}
			\includegraphics[width=\linewidth]{svc_synthetic_-7_crop.pdf}
			\caption{Synthetic}
			\label{fig:SVC_synthetic}
		\end{subfigure}
		\begin{subfigure}{0.245\linewidth}
			\includegraphics[width=\linewidth]{svc_kdd10_small_-7_crop.pdf}
			\caption{\textsf{KDD2010.small}}
			\label{fig:SVC_kdd10_small}
		\end{subfigure}
		\begin{subfigure}{0.245\linewidth}
			\includegraphics[width=\linewidth]{svc_rcv1_-6_crop.pdf}
			\caption{\textsf{RCV1.train}}
			\label{fig:SVC_rcv1_train}
		\end{subfigure}
		\begin{subfigure}{0.245\linewidth}
			\includegraphics[width=\linewidth]{svc_news20_-6_crop.pdf}
			\caption{\textsf{News20}}
			\label{fig:SVC_news20}
		\end{subfigure}
		\caption{Ablation study for the practical effect of sparse variance correction (abbreviated as SVC in the legends). Run 10 seeds. Shaded bands indicate $\pm 1$ standard deviation.}
		\label{fig:SVC}
	\end{center}
\end{figure*}
\begin{figure*}[t]
	\begin{center}
		\begin{subfigure}{0.245\linewidth}
			\includegraphics[width=\linewidth]{lu_synthetic_-7_crop.pdf}
			\caption{Synthetic}
			\label{fig:LU_synthetic}
		\end{subfigure}
		\begin{subfigure}{0.245\linewidth}
			\includegraphics[width=\linewidth]{lu_kdd10_small_-7_crop.pdf}
			\caption{\textsf{KDD2010.small}}
			\label{fig:LU_kdd10_small}
		\end{subfigure}
		\begin{subfigure}{0.245\linewidth}
			\includegraphics[width=\linewidth]{lu_rcv1_-6_crop.pdf}
			\caption{\textsf{RCV1.train}}
			\label{fig:LU_rcv1_train}
		\end{subfigure}
		\begin{subfigure}{0.245\linewidth}
			\includegraphics[width=\linewidth]{lu_news20_-6_crop.pdf}
			\caption{\textsf{News20}}
			\label{fig:LU_news20}
		\end{subfigure}
		\caption{Running time comparison between using sparse gradient estimator and lagged update (abbreviated as LU in the legends). The wall-clock time and objective value are averaged over 10 runs. }
		\label{fig:LU}
	\end{center}
\end{figure*}
\begin{figure*}[t]
	\begin{center}
		\begin{subfigure}{0.27\linewidth}
			\includegraphics[width=\linewidth]{kdd10_-10_20t_2_3_crop.pdf}
			\caption{\textsf{KDD2010}, $20$ threads}
			\label{fig:AS_kdd10}
		\end{subfigure}
		\quad\quad
		\begin{subfigure}{0.27\linewidth}
			\includegraphics[width=\linewidth]{rcv1_-9_20t_1_1d3_crop.pdf}
			\caption{\textsf{RCV1.full}, $20$ threads}
			\label{fig:AS_rcv1}
		\end{subfigure}
		\quad\quad
		\begin{subfigure}{0.27\linewidth}
			\includegraphics[width=\linewidth]{avazu_-10_20t_2_2_crop.pdf}
			\caption{\textsf{Avazu-site.train}, $20$ threads}
			\label{fig:AS_avazu}
		\end{subfigure} \\
		\,\,
		\begin{subfigure}{0.27\linewidth}
			\includegraphics[width=0.9\linewidth]{kdd10_speedup-crop.pdf}
			\centering
			\caption{\textsf{KDD2010}, speed-up}
			\label{fig:SU_kdd10}
		\end{subfigure}\quad\quad\,
		\begin{subfigure}{0.27\linewidth}
			\includegraphics[width=0.9\linewidth]{rcv1_speedup-crop.pdf}
			\centering
			\caption{\textsf{RCV1.full}, speed-up}
			\label{fig:SU_rcv1}
		\end{subfigure}\quad\quad\,
		\begin{subfigure}{0.27\linewidth}
			\includegraphics[width=0.9\linewidth]{avazu_speedup-crop.pdf}
			\centering
			\caption{\textsf{Avazu-site.train}, speed-up}
			\label{fig:SU_avazu}
		\end{subfigure}
		\caption{Convergence and speed-up for asynchronous sparse methods. Speed-up is the improvement on the wall-clock time to achieve $10^{-5}$ sub-optimality relative to using a single thread.}
		\label{fig:AS_SU}
	\end{center}
\end{figure*}

\subsection{Asynchronous Experiments}
We compare the practical convergence and speed-up of AS-Acc-SVRG (Algorithm \ref{alg:AS-Acc-SVRG}) with KroMagnon and ASAGA in Figure \ref{fig:AS_SU}.  We do not compare with the empirical method MiG in \citep{MiG}, which requires us to tune two highly correlated parameters with only limited insights. This is expensive or even prohibited for large scale tasks. For the compared methods, we only tune the $\tau$-related constants in their theoretical parameter settings. That is, we tune the constant $1 + 2\sqrt{\Delta}\tilde{\tau}$ in Theorem \ref{thm:AS-Acc-SVRG} for AS-Acc-SVRG and the constant $c$ in the step size $\frac{1}{cL}$ for KroMagnon and ASAGA. In fact, in all the experiments, we simply fixed the constant to $1$ for AS-Acc-SVRG (the same parameters as the serial case), which worked smoothly. The main tuning effort was devoted to KroMagnon and ASAGA, and we tried to choose their step sizes as large as possible. The  detailed choices can be found in Appendix \ref{app:exp}.  Due to the scale of the problems, we only conducted a single run. From Figure~\ref{fig:AS_SU}, we see significant improvement of AS-Acc-SVRG for ill-conditioned tasks and similar practical speed-ups among the three methods, which verifies Theorem \ref{thm:AS-Acc-SVRG} and Corollary \ref{cor:speed-up}. We also observe a strong correlation between the practical speed-up and $\Delta$, which is predicted by the theoretical $O(1/\sqrt{\Delta})$ dependence. It seems that we cannot reproduce the speed-up results in \citep{ASAGA_JMLR} on \textsf{RCV1.full} dataset. We believe that it is due to the usage of different programming languages, that we used a low-level heavily-optimized language C\nolinebreak\hspace{-.05em}\raisebox{.4ex}{\tiny\bf +}\nolinebreak\hspace{-.10em}\raisebox{.4ex}{\tiny\bf +}\,and they used high-level language Scalar. \citet{ProxASAGA} also used C\nolinebreak\hspace{-.05em}\raisebox{.4ex}{\tiny\bf +}\nolinebreak\hspace{-.10em}\raisebox{.4ex}{\tiny\bf +}\,implementation and we observe a similar $10\times$ speed-up of ASAGA on \textsf{KDD2010} dataset.

\section{Conclusion}
In this work, we proposed a new asynchronous accelerated SVRG method which achieves the optimal oracle complexity under the perturbed iterate framework. We show that it requires the same linear speed-up condition as the non-accelerated methods. Empirical results justified our findings.

A limitation of this work is that it does not support proximal operators, and we are currently not sure how to leverage the techniques in \citep{ProxASAGA} into our method. The main difficulty is on the choice of potential function.

Future directions could be (i) a more fine-grained analysis on $\Delta$ since currently $\Delta$ is a constant in all the datasets (except for the synthetic one), and (ii) designing asynchronous accelerated SAGA based on \citep{SSNM} since SVRG variants still require at least one synchronization per epoch.

redpajama_set_name: RedPajamaArXiv


Residents were being asked to keep inside their houses. Please include a contact number if you're prepared to talk to some sort of BBC journalist. There was essential to achieve more perfect means to respond to this type of news. Stopping of which fraud is currently a priority, mentioned previously by the IRS.
Gene therapy trials regarding Usher syndrome are in their very own first stages, and we have hope they could save my hearing and vision afterward. I decided this case for a couple of law. You carried a party of Rip-Its.
It isn't humane, you cannot even feel as an personal, you have wires all over your whole body, " explained Msumba. The woman managed to hold on to the woman keys also. Just a few family members knew. He's going be published in 2019. This is nowhere near the ending. It doesn't do the job, " the girl states. There are those who have an issue with a few of the things Donald Overcome has said, and a few of the insurance policies he has not articulated.
Do make excuses when you are dealing with real bodily symptoms, or perhaps watch for a medical crisis before you find a health care provider. Nowadays normally the prognosis for this isn't very encouraging, although I'm likely to fight this particular and I will keep working, he explained. For more advanced forms of pancreatic cancer, chemotherapy is generally used, Dr . Jacoub reports. There isn't any other treatment alwi.up-dated.info for those persons here that suffer from these kinds of severe behavior disorders. I would give this pastor the main advantage of the doubt because he has got a potent ministry. My partner and i wished to make certain that I got to my entire potential. This lady has the capability to be a voice for a number of of the other children who are not able to speak for themselves.
The driver's side front door closes as there seems to certainly be a struggle in the motor vehicle. I just felt the same as a bit of garbage. The series did not discover returning. There is a good deal great that comes from the show, and also lots of negativity. However , I feel like I am only able to take care of this show a person season at a moment. It's unusual in order for it to be diagnosed at a young stage. Listen, we're likely to cover the race fairly, we always have in the first place.
We are likely to ensure they do the things which they claim they're prone to do, " George described. It's alleged they present severe burns Andre experienced as a consequence of the shocks. Your lover breathed a fast prayer that life may be long.


From last year to this year, most draft evaluators already know about the quarterback class. What can you takeaway from it? It's already better than the last years. Plenty of question marks surrounded the previous draft class but this current one is fire. That's what the kids call it nowadays, right?
Many pre-season rankings for college football already have Sam Darnold as the top quarterback with a combination of Josh Allen, Luke Falk, Josh Rosen and Mason Rudolph behind him. What falls behind those five quarterbacks though? There will be names such as Jake Browning, Baker Mayfield and Lamar Jackson. Another name that you need to know is Ryan Finley from North Carolina State.
He's listed at 6'4″ and 205 lbs., from Phoenix, Arizona. Originally, he committed to Boise State but transferred in the summer of 2016. During that time, he was granted a sixth-year of eligibility and also earned his degree after only three years.
Last year was his first season at North Carolina State and it was a solid one, at that. While the Wolfpack only won six games, the redshirt sophomore quarterback completed over 60% of his passes. By doing so, he compiled 3,059 yards and 18 touchdowns. It also took him 139 passing attempts before throwing his first interception on the season. Ultimately, he finished with 8 interceptions on the season and was throwing one interceptions per 50.25 pass attempts.
Many times, you'll see Ryan Finley out of a spread offense. Whether it's an empty formation (no running backs, just the quarterback in the backfield) or trips left (3 wide receivers to the left). There's a wide variety of packages you'll see out of the spread and the Wolfpack do a great job spreading the football around. From running the ball to passing the ball, they're certainly unpredictable. Altogether, the Wolfpack threw 438 times and had 504 rushing attempts on the season.
With this play, you'll notice the empty formation. Finley takes the snap and throws a quick strike between the numbers and hashmark. With this soft coverage, the play turns into a much larger gain than it's designed. It's a 7-yard throw and turns into more after the catch. On the season, Finley averaged 7.6 yards per pass attempt.
Another spread look from Finley and the Wolfpack. This is a designed screen pass to their H-back, Jaylen Samuels. H-backs and tight ends aren't supposed to move like this but this was great after the catch. With screen passes such a big part of the NFL, Finley will excel in a designed system with playmakers like Samuels.
Before this specific play, Finley missed on two straight throws. He had two open targets and missed both times. However, it's third-down and he redeems himself. His progression in the pocket looks natural and he looks comfortable with a formidable pass rush to his right. While the receiver runs an out-route, the timing of it couldn't be better. Finley fires a strike to his left and a first-down is gained.
Much like every quarterback, not every play is going to be perfect. Finley throws a lot of quick passes but at times, he'll take shots down field. Here's a deep pass to his right that isn't terribly placed but it's not great either. There was great coverage on the play and the receiver had no shot of adjusting to the football. Ideally, this ball should be placed higher with it being a 50/50 throw but there wasn't enough air under it for that to happen.
After watching four games from Ryan Finley, this was a play that stood out. It was against North Carolina and it demonstrates what he possesses for a deep ball. Above, you'll see twins right and both receivers will be running vertical.
Throughout my pre-season breakdown on Ryan Finley, I found a variety of strengths and weaknesses. Some of his strengths are simple. He has the size at 6'4″ and doesn't seem to struggle reading defenses. With a spread offense, it forces teams to spread out defensively and not stack the box. This becomes his bread and butter with his 7.6 yards per passing attempt. He displays a strong arm with his quick strikes and doesn't struggle with getting the ball down field.
With that being said, there's certainly some ball placement issues. He doesn't always lead his receivers and doesn't always get enough air on the ball. This will lead to some mistakes from interceptions and deflections from the defense.
Without playing a down for the 2017 season, Ryan Finley is a quarterback with an early fourth round grade. If he has a strong season, he has the upside to hear his name in the second round. The only way to find out is by tuning into the Wolfpack and their action packed schedule. They kick their season off against South Carolina on September 2nd, 2017 in Charlotte, North Carolina.


text:
Q: How to get the url with freemarker template? I want to fetch current URL with free marker.
Also i need to split the url href part.Is there any method similar to JavaScript method location.href.
Ex: http://www.barcamp.com/homepage/overview.go.
Here i need to fetch the url & also to the split the url and get its as /homepage/overview.go
Please help us on this issue

A: Getting the current URL falls outside FreeMarker's jurisdiction, since it's independent of Servlets and HTTP. It's up to the Web application framework if and how it exposes that information for the templates, so check the documentation of the framework.

redpajama_set_name: RedPajamaStackExchange


text:
Papá por un día (Spanish: Dad for a Day) is a 2009 Argentine sports romantic comedy film starring Nicolás Cabré, Luisana Lopilato and Gimena Accardi. It was premiered on 6 August 2009, and was also screened at the Mar del Plata Film Festival on 14 November 2009, not as the part of the competition.

Plot
Federico Villaverde (Nicolás Cabré) is the coach of the female hockey team of a very high-level club. He is engaged to Cecilia (Gimena Accardi), one of the players and the daughter of Luciano Ayerza (Boy Olmi), the owner of the club; when his dying father (Gustavo Garzón), whom he had not seen in ten years, asks him to take care of Tini (Julieta Poggio) his 8-year-old sister, whom he has never met before, things get upside down for him as he tries to adjust this new life. Amidst this situation he meets Julieta (Luisana Lopilato), the captain of the hockey team his father used to coach, and unexpected feelings arise.

Cast
 Nicolás Cabré as Federico Villaverde
 Luisana Lopilato as Julieta Miriotti
 Gimena Accardi as Cecilia Ayerza
 Gustavo Garzón as Eliseo Villaverde
 Gogó Andreu as Lorenzo
 Inés Palombo as Teresa
 Patricia Sosa as Beba
 Nicolás Vázquez as Taxi driver
 Julieta Poggio as Martina "Tini" Villaverde
 Boy Olmi as Luciano Ayerza
 Miguel Ángel Rodríguez as Tito Miriotti
 Joaquín Berthold as Mariano Alerce
 Martín Borisenko as Gonzalo de la Oz
 Mónica Gonzaga as María Inés
 Roberto Dolade as Doctor
 Carlos Kaspar as Kit manager
 Bárbara Diez as Wedding planner

Reception

Critical response
The newspaper La Nación described the film as "Good", noting that "The plot achieves what it set out to do: to entertain with successful gags, funny dialogues and situations."

While the film critic Ezequiel Boetti from Escribiendo Cine rated it with a score of 4 out of 10, describing it as "a cinematic concoction that includes from sports competitions to the worst clichés in Argentine films.", "an ad infinitum concatenation of the worst things in local filmmaking." and negatively highlighted "the thick writing with which the scriptwriters portrayed the two female leads".

Also 4 out of 10 was the rating of Diego Battle from Otros Cines, who pointed out that "the script by Jorge Maestro is quite disappointing, inferior than -to name a few other works by its author- products like Pelito, La Banda del Golden Rocket or Montaña Rusa. In this very weak film stereotypes, caricatures, simplifications, superficial things and clichés abound. The plot is uninspired, there is not much grace, neither emotion, nor sensitivity, neither humanity nor audacity."

For its part, the newspaper Clarín described it as "Average".

Box office
The film reached 480,568 viewers, reaching 20th place in the 2009 Argentine box office ranking. Dad for a Day grossed a total of $1.866.255 million.

References

External links

2009 films
2000s sports comedy films
2000s Spanish-language films
2009 romantic comedy films
Argentine sports comedy films
Argentine romantic comedy films
2000s Argentine films


After some time of not using the PC the screen turns off for power saving. When moving the mouse to resume the screen stays black with only the moving mouse cursor.
made it executable and redirected the /etc/X11/X link to it. I have also installed valgrind, xserver-xorg-core-dbg, xserver-xorg-input-evdev-dbg, xserver-xorg-input-synaptics-dbg, xserver-xorg-video-intel-dbg.
After that I have logged out and logged in again so that X runs under Valgrind.
The problem does not occur very often, only every few days. As soon as it happened I switched to a text console and first copiied the log file from vg.till to vg.till.1 (attached) and then ran "sudo apport-bug xorg" to create this bug report (the command starts a text mode browser).
The log file vg.till.1 is attached.
I have added "--error-limit=no" to the valgrind commend line in /etc/X11/X2 as valgrind told that it stopped reporting shortly before I have observed the problem the first time with X under Valgrind (comment #2). Then I have restarted X and waited for the problem to show up again which happened now. I copied vg.till to vg.till.2 and attached it to this comment.
Problem occured again, valgrind log under the same conditions as described in comment #3 attached.
Right after my last comment I did a system update which included said version of xserver-xorg-video-intel. I will keep observing for some days and tell whether the problem has gone away.
It seems that the problem is fixed now as it does not occur any more for me. As running X under Valgrind slows down X significantly I have turned off the Valgrind observation again.


If you look closely at most others' dies, you can see somewhat of a rough finish on the face of the dies, which can adversely affect their use.
So cool to be able to wear a quarter as jewelry!
There is a rich history surrounding coin rings. Many of the first ones were a product of difficult times of war and or economic difficulties.
Annealing Coin RIngs - How Hot is too Hot?
How to make coin rings from start to finish!
A few tips on how to prevent marring up the inside of your coin rings.
How to make a double sided coin ring- different holes make different rin.
The plastic resin ball bearings like the ones shown above were used along with a reduction die to achieve a folded-over coin, such as the one shown below.


text:
Looking back: Margaret Moore
Some months have passed since the Into The Mountain performances and with this we have invited the amazing team of women who made it happen to share their reflections, thoughts and feelings after the event. From production to performing, guiding audiences and behind the scenes, all have taken something away from the collective experience of Into The Mountain into their daily lives and practices.
Here, a member of the choir, Margaret Moore, shares the process of singing with Lucy, Hanna, and the rest of the group:
"I was in the choir singing the score being recorded for the film of "Into the Mountain" in Kincraig Village Hall on August 10, 2019. It's a pleasant hall, new with a lot of wood. The plan was to practise for a couple of hours, record the score, have lunch and then go out into the Cairngorms to sing there. Rain was forecast.
We warmed up, a semicircle around Laura and Hanna. The score begins with "ahs" and "yo-do-dos". I was doing the "Ahhs", where you can flow between A, G, F or stay on one note. No problem. But it was a problem when we started, in Aviemore, in another beautiful hall. My singing experience is all about staves, notes, bar lines. For a while I was totally at sea with "Ahhs" and "Yo-do-do's" and so on flowing across the page. When to start, when to stop, what do people want? But it did make a lovely sound, so I stayed.
The tricky bit of the score is the second part, the rhythmic part. It starts with Hanna and Laura flinging what sounds like a melodic shriek into the air and then we – Sandy, me and Catherine – come in with "he-u-we -ho", A, G, F, Catherine lagging. We take the time from Hanna and Laura and we keep it going. The other parts come in on top. Lagging means coming in a beat or two after the start. I spent the best part of an afternoon in Aviemore trying to do lagging, could not get the hang of it. Catherine loves it, thank goodness for that. But we are too driving, Hanna says. Embody the sound. Right. So we bounce it a bit. We do it again. And again. And again. "You're working", she says now, but still the piece is not right so we do it over and over and over. I joked once that us He-u-we-ho's were the Charlie Watts of the piece. How did Charlie do it? In a lull I make a run for an energy bar in my bag.
The rain didn't come, it's sunny outside, heating up in the hall. The piece is still not right, we break for lunch. Back after a pleasant time sitting on the grass in the sun. We go on to the "outro", the part that ends the rhythmic section and it is recorded in one take. Then the next section – C – also done quickly. Back to section B. All OK, finally. The sun is shining through the hall windows. The last section is the best, I think. It is a re-arrangement/re-interpretation – not sure how to put it – of a traditional song "Up a Wide and Lonely Glen". Part of our preparation was to listen to Jeannie Robertson's 1953 recording of it on the Tobar an Dualchais site. She had it from her mother, who came from Ballater. We sing words and phrases from this, but it's one of those cases where words are useless to describe what happens. We're singing – Julia, Angela and me this time – "there di I spy", "lonely", "fare thee well". The score is colour coded, we're pink, high voices. Hanna's bringing in the high voices, Laura the lower. We do it several times. Gradually I bear in on it. Shape the words. Pay attention to endings. Put all I can into our first "Fare the well" – the top of the piece, so to speak – lower and sadder for the second.
Rain finally comes and the change in temperature makes the roof creak. Difficult for recording , but it's done. We're going out to sing. A spot near the ski lift. I go to the car to get my walking boots on ready, then realise I'm going to have to drive so better not. My brain's mushy – heat. The walking is a Big Thing far as I'm concerned. I went to two rehearsals up at the performance site but then couldn't do the actual performances, which led to weeks chuntering in my head about getting old, giving up versus falling over, etc. etc.
Back in the hall they are waiting for me. We're doing "Shh" – passing breath from one to another. In the middle of the recording I want to cough, I'm bent over, just hold it until the finish.
I follow the car in front, don't know where the ski lift is. I look down and realise I'm driving in my socks. At the car park near the lift we are attacked with midges. Julia's got cream, gives me some. We are to walk a short way up a little brow – you could measure it in yards. I go up, huff, huff, huffing. Really. Midges swirling everywhere. Can't sing for midges. Have to come down, find another spot. We find it looking out over Cairngorm. It is suddenly cooler. My brain clears. No midges. We sing. A song sung by Jeannie and many others, re-made now, and I fling my "Fare thee well" out over the land it came from. And that was it."
Looking back: Norma D. Hunter