text:
50 Most Underrated NBA Players of All Time
None of these underrated NBA players played in more than two All-Star Games, and they all flew well under the radar.
The problem with most underrated NBA lists is too many of the players are, well, overrated.
A lot of the same names show up even though they were perennial All-Stars. (Here's looking at you, Jack Sikma.) Some even starred in major markets. (Isn't that right, Maurice Cheeks and Bobby Jones?)
Heck, in some cases, a guy has been underrated by so many for so long, he's become a bit overrated. (Not to name names or anything, Sidney Moncrief.)
Now here's the real list of all-time underrated NBA players. None of them played in more than two All-Star Games, and most flew well under the radar.
50. J.J. Redick
Position: Guard
Career: 17 seasons (1984-2001)
Teams: Orlando Magic (206-13), Milwaukee Bucks (2013), Los Angeles Clippers (2013-17), Philadelphia 76ers (2017-18), New Orleans Pelicans (1919-21), Dallas Mavericks (2021).
All-Star Games: 0
League championships: 0
Bottom Line: J.J. Redick
Whether you loved to hate Redick or hated to love him, the ex-Dookie could shoot the rock.
He didn't steal any paychecks either.
49. Otis Thorpe
Position: Forward-center
Teams: Kansas City-Sacramento Kings (1984-88, 1998), Houston Rockets (1988-95), Portland Trail Blazers (1995), Detroit Pistons (1995-97), Vancouver Grizzlies (1997-98), Washington Wizards (1998-99), Miami Heat (1990-2000), Charlotte Hornets (2000-01)
All-Star Games: 2001
Bottom Line: Otis Thorpe
Otis our man performed the pick-and-shovel work about as well as anyone for the better part of two decades.
He ranks among the top 40 in minutes played, field goal percentage, offensive rebounds and defensive rebounds in league history.
48. Hot Rod Williams
Career: 13 seasons (1986-99)
Teams: Cleveland Cavaliers (1986-95), Phoenix Suns (1995-98), Dallas Mavericks (1998-99)
Bottom Line: Hot Rod Williams
Williams was the most overlooked player with multiple Cavaliers playoff teams.
The 6-foot-11 big was a dependable rebounder and double-figure scorer as well.
47. Toni Kukoc
Position: Guard-forward
Teams: Chicago Bulls (1993-2004), Philadelphia 76ers (2000-01), Atlanta Hawks (2001-02), Milwaukee Bucks (2002-06)
League championships: 3 (2006-08)
Bottom Line: Toni Kukoc
Oh, how Scottie Pippen and Michael Jordan hated Kukoc for the way that GM Jerry Krause fawned over him.
The Croatian was no sensation, but he was a valuable movable piece with three championship teams.
46. Terry Cummings
Teams: San Diego Clippers (1982-84), Milwaukee Bucks (1984-89), San Antonio Spurs(1989-95), Seattle SuperSonics (1996-97), Philadelphia 76ers (1997-98), New York Knicks (1998), Golden State Warriors (1999-2000)
All-Star Games: 2 (1985, 1989)
Bottom Line: Terry Cummings
As one of the top 50 career scorers in NBA history, Terry Cummings deserves a spot on this list. Throughout his 18-year career, he scored a total of 19,460 points, just short of the honorable 20,000-point mark.
Fun fact: Cummings actually grew up wanting to be a professional hockey player, but a high school growth spurt steered him into basketball.
45. Darryl Dawkins
Career: 1975-89 (14 seasons)
Teams: Philadelphia 76ers (1975-82), New Jersey Nets (1982-86), Utah Jazz (1987), Detroit (1987-89)
Bottom Line: Darryl Dawkins
Dawkins was one of those bigs who could have used college ball to develop his raw but considerable game. He entered the Association at 19 years of age.
"His talent never really got a chance to shine with us," ex-teammate Doug Collins once said. "He was the first to do what he was doing, and I think he deferred at times."
44. Al Jefferson
Position: Center-forward
Teams: Boston Celtics (2004-2007), Minnesota Timberwolves (2007-10), Utah Jazz (2010-13), Charlotte Bobcats-Hornets (2013-16), Indiana Pacers (2016-18)
Bottom Line: Al Jefferson
Jefferson was one of the better players never to appear in an All-Star Game or NBA Finals.
He produced three seasons of 21 points and 10 rebounds per game.
43. Alvan Adams
Teams: Phoenix Suns
Bottom Line: Alvan Adams
Statistically, Adams had his best season as a Rookie of the Year. It could be that it led to unrealistic expectations.
Whatever the case, the Suns lifer was on the shortlist of better all-around pivotmen for an extended period.
42. Rashard Lewis
Teams: Seattle SuperSonics (1998-2007), Orlando Magic (2007-10), Washington Wizards (2010-12), Miami Heat(2012-14)
Bottom Line: Rashard Lewis
Lewis was Kevin Durant Lite — a 6-foot-10 swing forward who could hit the longball.
A 20-points-per-game scorer in three seasons, Sweet Lew ranks 22nd in career 3-point percentage.
41. Larry Johnson
Teams: Charlotte Hornets (1991-96), New York Knicks (1996-2001)
Bottom Line: Larry Johnson
To those who say that L.J. was a disappointment, we say buzz off. Have we forgotten how he helped turn the Hornets into a legit playoff contender in only two years?
He averaged 19.6 points, 9.2 rebounds and 4.1 assists per game in five seasons there.
40. Ed Pinckney
Teams: Phoenix Suns(1985-87), Sacramento Kings (1987-89), Boston (1989-94), Milwaukee Bucks (1994-95), Toronto Raptors (1995-96), Philadelphia 76ers (1996), Miami Heat (1996-97)
Bottom Line: Ed Pinckney
Pinckney learned how to win in his Villanova days.
That translated well at the next level, where the combo forward did enough to carve a niche for himself.
39. A.C. Green
Teams: Los Angeles Lakers (1985-93, 1999-2000), Phoenix Suns (1993-96), Dallas Mavericks (1996-99), Miami Heat (2000-01)
League championships: 3 (1987, 1988, 2000)
Bottom Line: A.C. Green
Green almost never took a night off (he played in 1,192 games at one point), swore off booze even at championship celebrations and claimed to be a virgin well into his 30s.
No way could the guy play in the Association today.
38. Archie Clark
Teams: Los Angeles Lakers (1966-68), Philadelphia 76ers (1968-71), Baltimore/Capital Bullets (1971-74), Detroit Pistons (1974-75), Seattle SuperSonics (1975-76)
Bottom Line: Archie Clark
Clark wrote the book on Shake and Bake. Literally. He also gave us the step-back jump shot.
For that, the man deserves a place in the Hall of Fame contributors wing at least.
37. Michael Redd
Teams: Milwaukee Bucks (2000-11), Phoenix Suns (2011-12)
Bottom Line: Michael Redd
There was no backcourt player more fun to watch than Redd in the mid-2000s.
This is when he put up 23.5 points per game over five seasons.
36. Hersey Hawkins
Teams: Philadelphia 76ers (1988-93), Charlotte Hornets(1993-95, 2000-01), Seattle SuperSonics (1995-99), Chicago Bulls (1999-2000)
Bottom Line: Hersey Hawkins
A career 46/39/87 stat line puts Hawkins in an elite class of all-time shooters.
35. Clifford Ray
Teams: Chicago Bulls (1971-74), Golden State Warriors (1974-81)
League championships: 1975
Bottom Line: Clifford Ray
It was no coincidence that, wherever Ray manned the middle, his team was among the league leaders in opponents' field goal percentage.
The Mule was an All-Star-caliber defender-rebounder even if he never played in the showcase event.
34. Kevin Martin
Teams: Sacramento Kings, (2004-09) Houston Rockets (2009-12), Oklahoma City (2012-13), Minnesota Timberwolves (2013-16), San Antonio Spurs (2016)
Bottom Line: Kevin Martin
When the 3-pointer took off in the mid-2000s, Martin was at the front of the line.
K-Mart drained at least 36 percent of his longballs in nine seasons and surpassed the big four-oh on three occasions.
33. Marcus Camby
Teams: New York Knicks (1998-2001, 2012-13), Denver Nuggets (2002-08), Los Angeles Clippers (2008-10), Portland Trail Blazers (2010-12), Houston Rockets (2012)
Bottom Line: Marcus Camby
In any discussion about the best defenders in the game, Camby isn't among the first names to roll off most tongues. Maybe it should be.
The one-time Defensive Player of the Year ranks 13th in blocked shots and 35th in defensive rebounds on the all-time list.
32. Happy Hairston
Teams: Cincinnati Royals (1964-68), Detroit Pistons (1968-69), Los Angeles Lakers (1969-1975)
Bottom Line: Happy Hairston
How valuable was Happy (Harold) Hairston? On the heels of an epic 69-13 regular season and league title, the 1972-73 Lakers were even better.
They got off to a 24-4 start before Wilt Chamberlain's tag-team partner blew out a knee. They weren't the same without his 16 points, 13 rebounds and dogged defense each game.
31. Sam Perkins
Career: 17 seasons(1984-2001)
Teams: Dallas Mavericks (1984-90), Los Angeles Lakers (1990-93), Seattle SuperSonics (1993-98), Indiana Pacers (1998-2001)
Bottom Line: Sam Perkins
Perkins played with some pretty fair players in his career — Michael Jordan, Magic Johnson, Gary Payton and Reggie Miller to name a few — and some would tell that he was the victim of low expectations as a result.
Yet Sleepy Sam was a dependable scorer and rebounder for much of his career. As his game moved away from the bucket, he became the rare big who could shoot the 3-ball.
30. Anthony Mason
Teams: New Jersey Nets (1989-90), Denver Nuggets (1990-91), New York Knicks (1991-96), Charlotte Hornets (1996-2000), Miami Heat (2000-01), Milwaukee Bucks (2001-03)
Bottom Line: Anthony Mason
Hey, look — a 40-minutes-per-game player! Mason averaged as many over a five-year period.
Mase played hard and played tough and played at both ends. No wonder the human fireplug was popular wherever he played.
29. Zydrunas Ilgauskas
Teams: Cleveland Cavaliers
Bottom Line: Zydrunas Ilgauskas
Ilgauskas was the best of the LeBron James supporting cast who were swept in the 2007 NBA Finals. Which was sort of like being second to Arnold Schwarzenegger in the Mr. Universe contest.
But the 7-foot-3 big deserves some love here. The guy had all sorts of health issues — included were two fractures of the same bone in his left foot — but he overcame them to have a solid career.
28. Doc Rivers
Teams: Atlanta Hawks (1983-91), Los Angeles Clippers (1991-92), New York Knicks (1992-94), San Antonio Spurs (1994-96)
Bottom Line: Doc Rivers
Rivers did everything well except shoot the long ball.
The second-rounder spent his prime years in the shadow of teammate Dominique Wilkins, whom he fed for many a dunk.
27. Steve Smith
Teams: Miami Heat (1991-94, 2005), Atlanta Hawks (1994-99), Portland Trail Blazers (1999-2001), San Antonio Spurs (2001-03), New Orleans Hornets (2003-04), Charlotte Hornets (2004-05)
Bottom Line: Steve Smith
Smith was one of the most consistent scorers of the 1990s decade.
He averaged 17-plus points per game in seven consecutive seasons and put up a .440/.358/.845 percentage line on more than 15,000 shot attempts.
26. Nene Hilario
Teams: Denver Nuggets (2002-12), Washington Wizards (2012-16), Houston Rockets (2016-19)
Bottom Line: Nene Hilario
Durability was a challenge, but when healthy, Nene was an efficient scorer in the paint at one end and competent rebounder-defender at the other.
The 6-foot-11 big averaged at least 12 points and a half-dozen rebounds per game in six seasons.
25. Steve Kerr
Teams: Phoenix Suns (1988-89), Cleveland Cavaliers (1989-92), Orlando Magic (1992-93), Chicago Bulls (1993-98), San Antonio Spurs (1999-2003), Portland Trail Blazers (2001-02)
League championships: 5 (1996-99, 2003)
Bottom Line: Steve Kerr
Steph Curry may have more career 3-pointers, but Kerr shot them more accurately than anyone in league history.
He accepted his role and kept his mouth shut, which played no small role in his success.
24. Sam Cassell
Teams: Houston Rockets (1993-96), Phoenix Suns (1996), Dallas Mavericks (1996-97), New Jersey Nets (1997-99), Milwaukee Bucks (1999-2003), Minnesota Timberwolves (2003-05), Los Angeles Clippers (2005-07), Boston Celtics (2007-08)
Bottom Line: Sam Cassell
Sam I Am was so consistent for so long and with so many teams that he was taken for granted.
He ranks high on the all-time list in two important categories for point guards — assists (40th) and field throw percentage (47th).
23. Dale Davis
Teams: Indiana Pacers (1991-2000, 2005), Portland Trail Blazers (2000-2004), Golden State Warriors (2004-05), Detroit Pistons (2005-07)
Bottom Line: Dale Davis
Every contender needs a rebounder and interior defender who they can count on every night. That was Davis for two decades.
His teams advanced to the playoffs 14 of 16 seasons.
22. Marcin Gortat
Teams: Orlando Magic (2007-10), Phoenix Suns (2010-13), Washington Wizards (2013-18), Los Angeles Clippers (2018-19)
Bottom Line: Marcin Gortat
The Polish Hammer was among the top 20 in field goal percentage (six), rebounds (six) and blocked shots (three) multiple times.
His .551 field goal percentage ranks 24th in league history.
21. David West
Teams: New Orleans/Oklahoma City Hornets (2003-11), Indiana Pacers (2011-15), San Antonio Spurs (2015-16),Golden State Warriors (2016-18)
All-Star Games: 2 (2008-09)
Bottom Line: David West
At 6-foot-9, West was a valuable two-way movable piece who could do everything except hit the long ball.
He was among the select few who took a sizable pay cut to play a lesser role with a championship contender.
20. Mehmet Okur
Teams: Detroit Pistons (2002-2004), Utah Jazz (2004-11), New Jersey Nets (2011-12)
Bottom Line: Mehmet Okur
Okur had a game that was far more consistent than it was pretty.
The second-rounder was good for at least 10 points and a half-dozen rebounds every season.
19. Brent Barry
Teams: Los Angeles Clippers (1995-98), Miami Heat (1998), Chicago Bulls (1999), Seattle SuperSonics (1999-2004), San Antonio Spurs (2004-08), Houston Rockets (2008-09)
League championships: 2 (2005, 2007)
Bottom Line: Brent Barry
Barry was misplaced as a swingman. He had the tools to be an even better point guard — handle, instincts, athleticism, genetics and perimeter shot.
He still had a productive career, which featured contributions with two championship teams.
18. Carlos Boozer
Teams: Cleveland Cavaliers (2002–04), Utah Jazz (2004-10), Chicago Bulls (2010-14), Los Angeles Lakers (2014-15)
Bottom Line: Carlos Boozer
Critics were fond of the Boo in Boozer, but expectations were unrealistic. He put together seven seasons of at least 16 points and nine rebounds per game.
Only Yao Ming and Amare Stoudemire averaged more Win Shares per 48 minutes in the 2002 draft class.
17. Jeff Foster
Teams: Indiana Pacers
Bottom Line: Jeff Foster
So much did the Pacers value Foster off the bench, the career overachiever spent his entire time in Naptown.
The guy was a scrappy defender and underrated ball retriever, particularly at the offensive end, where he ranks sixth in rebound percentage in league history.
16. Calvin Natt
Position: Small forward and power forward
Teams: New Jersey Nets (1979-80), Portland Trail Blazers (1980-84), Denver Nuggets (1984-89), San Antonio Spurs (1989), Indiana Pacers (1989-90)
League championships: None
Bottom Line: Calvin Natt
On March 18, 1982, Calvin Natt pitched one of the few perfect games in NBA history. The Trail Blazer drained every one of his 13 shots (all field goals) in a win at Portland.
The pug-nosed forward looked and played the part for 11 seasons, all with mid-market teams. A tenacious rebounder for his 6-foot-6, 220 pounds, Natt combined fluid moves and a deft touch in the paint area. His offensive rating is 64th on the all-time list, ahead of James Worthy, Dominique Wilkins and Julius Erving, among others at the position.
Unfortunately, Natt was a so-called 'tweener — not quite athletic enough at small forward, not big enough to be the prototypical power forward.
A Natt's eyelash away from superstardom, one might say.
15. Ricky Pierce
Position: Point guard and shooting guard
Teams: Detroit Pistons (1982-83), San Diego Clippers (1983-84), Milwaukee Bucks (1984-91, 1997-98), Seattle SuperSonics (1991-94), Golden State Warriors (1994-95), Indiana Pacers (1995-96), Denver Nuggets (1996-97), Charlotte Hornets (1997)
Bottom Line: Ricky Pierce
Once upon a time, in an NBA not that long ago, there was something called a mid-range game. That's right — teams would look for opportunities inside the arc before the game morphed into so many 3-point shootouts. Few did if any did it better than Pierce for the better part of 16 seasons.
At 6-foot-4, Pierce was a compact package of savvy and touch, a nonstop series of jab steps, reloads, fade-aways, and drives that translated into 14.9 points per game on 49 percent in the field. He shot 87.5 percent at the free-throw line. In each of six consecutive seasons (1984-90), he shot 51 percent or better from the field, almost unheard of for a backcourt player. His offense rating ranks 46th overall.
Even though Pierce was on the shortlist of best sixth men in his prime, he was taken for granted too often. For one, Pierce attended a football school (Rice) that hadn't produced an NBA player in 14 years. He went on to play for eight teams, none in major markets. And it wasn't until he was 33 that he played in the conference finals for the first and last time.
14. Steve Mix
Career: 13 seasons (1969-1972, 1973-1983)
Teams: Detroit Pistons (1969-71), Philadelphia 76ers (1973-82), ABA Denver Rockets (1971), Milwaukee Bucks (1982-83), Los Angeles Lakers (1983)
Bottom Line: Steve Mix
At 26, Steve Mix was at the crossroads. The Detroit Pistons had cut the fifth-round draft pick only weeks into the 1972-73 season. Then he had a cup of coffee with ABA's Denver Rockets. No, make that a bagel bite. It lasted exactly four minutes.
Mix never could take a hint, though. After one season in the CBA, where his 31.1 points per game against a bunch of stiffs did wonders for his confidence, the swing forward made good on a tryout with the Philadelphia 76ers. Yep, those Sixers, the ones who finished a pathetic 9-73 the previous season.
Julius Erving referred to his vertically challenged roommate as "Sky" for obvious reasons. Indeed, Mix was the ultimate grinder, a 6-foot-7 bundle of smarts and hustle whose game had to be watched regularly to be appreciated. He could spot the open man, drain a flat-footed shot. He ran the floor deceptively well. He excelled at team defense, and few played the passing lanes better. And, oh, how the guy loved to compete.
That the Sixers reached the NBA Finals four times with Mix in the rotation said a lot about one of the biggest overachievers of his time.
13. Terrell Brandon
Teams: Cleveland Cavaliers (1991-97), Milwaukee Bucks (1997-98), Minnesota Timberwolves (1999-2002)
Bottom Line: Terrell Brandon
An argument can be made for Brandon as the best player not named LeBron James in Cavaliers history. He practically carried them to a team-record five consecutive postseason appearances.
He also ranks high on the all-time list of league leaders in assists (58th) and steals (71st) per game.
12. Horace Grant
Orlando Magic forward Horace Grant, left, gets set to shoot against the Boston Celtics in a 1995 playoff game.
Teams: Chicago Bulls (1987-94), Orlando Magic (1994-99, 2001-02), Seattle SuperSonics (1999-2000), Los Angeles Lakers (2003-04)
League championships: 4 (1991, 1992, 1993, 2001)
Bottom Line: Horace Grant
What set the Bulls apart in their heyday was pressure defense. Or the Dobermans, as coach Johny Bach called his three attack dogs Scottie Pippen, Michael Jordan and Horace Grant.
Ho, Ho, Ho, it's magic, you knooooowww.
Yeah, that's him — Horace Grant.
"Horace was the most overlooked part of our defense," Bach once told me. "He was required to cover a lot of ground at the back end, and there wasn't another big man with his length and mobility who could do it as well as he did."
At 6-foot-10, Grant was a solid rebounder-defender who could knock down the mid-range shot. Phil Jackson was reminded of the latter in the 1995 playoffs, not long after Grant signed with the Magic as a free agent. While the Bulls coach stubbornly double downed on Shaquille O'Neal in the low post, his former whipping boy drained open shot after open shot.
As Bach predicted before the 1992 draft, Grant would become a special player, albeit an undervalued one.
11. Kiki Vandeweghe
Teams: Denver Nuggets (1980-84), Portland Trail Blazers (1984-88), New York Knicks (1989-92), Los Angeles Clippers (1992-93)
Bottom Line: Kiki Vandeweghe
Vandeweghe was a drop-dead good at the perimeter, and a quick first step allowed him to beat his man off the dribble.
So what if the rest of his game was iffy at best? Pure shooters are at a premium, and for six consecutive seasons (1981-87), the points machine filled it up with the best of 'em.
10. Brad Miller
Teams: Charlotte Hornets (1998-2000), Chicago Bulls (2000-02, 2009-10), Indiana Pacers (2002-03), Sacramento Kings (2003-09), Houston Rockets (2010-11), Minnesota Timberwolves (2011-12)
All-Star Games: 2003, 2004
Bottom Line: Brad Miller
Supposedly, Brad Miller wasn't good enough to play major college ball, let alone professionally. Yet somehow, the Purdue product banged/clawed/elbowed/scratched/outworked his way through 14 seasons with a half-dozen teams, most of them not very good and in smaller cities.
The Indiana country boy played like he dressed — all flannels and jeans — but he did a lot of things well below the rim for a long time. A combative rebounder who packed a mid-range game, he was agile enough to finish a fast break on occasion. Few centers could deliver the ball better from the high post.
As Miller put it later in his career, "My body ain't worth spit, but I've got a heart."
Miller fun fact: He led the league in offensive rating in the 2004-05 season, when he produced a 15.6/9.6/3.9/1.2 slash line on 52 percent in the field and 81 at the charity stripe.
Miller overcame long odds to play as long as he did, and he can thank Shaquille O'Neal in part. In 2002, the bigs were involved in a bruhaha, and just when a stark-raving Shaq was ready for the kill, right fist cocked, he thought better of it. Or else Miller might have become the first on-court fatality in league history.
9. Terry Porter
Position: Point guard
Teams: Portland Trail Blazers (1985-95), Minnesota Timberwolves (1995-99), San Antonio Spurs (1999-2002)
Bottom Line: Terry Porter
Forget Blazermania and the one-hit wonders of 1977. For extended excellence, the Trail Blazers teams of the early 1990s were the best to set foot in Portland. While future Hall of Famer Clyde "The Glide" Drexler dominated highlight reels, Porter served as the heartbeat of a team that averaged 57.5 wins over a four-year period and included an NBA Finals appearance.
When Porter didn't run the attack — he averaged at least eight assists per game in five consecutive seasons — the career 46-percent shooter turned scorer at the off guard spot. In the process, the Moms Mabley look-alike elevated the combo guard position to another level.
No matter. Only after the 1990-91 season did Porter receive MVP consideration, and then only barely. He finished a distant ninth with 34-year-old geezer Larry Bird in the vote.
8. David Lee
Teams: New York Knicks (2005-10),Golden State Warriors (2010-15), Boston Celtics (2015-16), Dallas Mavericks (2016), San Antonio Spurs (2016-17)
Bottom Line: David Lee
Lee was a solid rebounder, particularly at the defensive end, and he packed solid mid-range game as well.
He led the league in Offensive Rating (points produced per 100 possessions) in the 2006-2007 campaign.
7. Andrei Kirilenko
Position: Power forward and small forward
Experience: 2001-15
Career: 13 seasons 2001-15
Teams: Utah Jazz (2001-11), Minnesota Timberwolves (2012-13), Brooklyn Nets (14-15)
Bottom Line: Andrei Kirilenko
No Russia-born player scored more points (9,431) in his NBA career. And none had a better-extended run than Andrei Kirilenko did with the Utah Jazz for 10 seasons, when he averaged 12.4 points, 5.6 rebounds, 2.0 blocked shots and 1.4 steals per game.
Sad to say, it was until late in his career that Kirilenko escaped from Salt Lake City, where NBA careers are known to die. Until then, AK-47 might as well have been back in Izhevsk, since his all-around game received so little notice around the league.
6. John Drew
Atlanta Hawks forward John Drew soars towards the basket against the Philadelphia 76ers in 1980.
Teams: Atlanta Hawks (1974-82), Utah Jazz (1982-84)
Bottom Line: John Drew
John Drew would become known as much for his cocaine addiction off the court as his dominance on it. That's too bad. The second-round draft pick out of tiny Gardner-Webb had game even if we saw too little of it.
Drew ranks 54th in player efficiency rating, the highest of any non-Hall of Famer in his era. The combo forward made a living off the offensive glass, where his quick ups and dogged pursuit translated into a crazy 42 percent of his rebounds.
Players make their names in the postseason, though, and Drew spent too many them on his couch. He took part in only 29 playoff games and never advanced beyond the conference semis.
Drew scored his biggest victory after retirement, it turned out, when he beat his toughest foe.
5. Jeff Hornacek
Position: Shooting guard and point guard
Teams: Phoenix Suns (1986-92), Philadelphia 76ers (1992-94), Utah Jazz (1994-2000)
Bottom Line: Jeff Hornacek
Why isn't this son of a coach in the discussion about the greatest pure shooters of all time?
In 14 seasons, Jeff Hornacek shot 50 percent in the field, 40 from beyond the arc and 88 at the free throw line. Among guards, his true shooting percentage (.582) ranks 16th overall. He also is 45th in postseason win shares.
The owner of one of the more creative free-throw routines around — before each shot, Horny wiped his right cheek three times as a way to say hello to his kids — he had basketball smarts that made up his lack of outrageous athleticism. He made an unforced error about as often as he bricked an open shot.
It could be that Hornacek's efficiency and drop-dead consistency was too boring for his own good.
4. Frank Ramsey
Position: Shooting guard and small forward
Career: 9 seasons (1954-64)
Teams: Boston Celtics (1954-64)
League championships: 7 (1957, 1959-64)
Bottom Line: Frank Ramsey
Before John Havlicek, there was Frank Ramsey, the Celtics' swingman whose standard for sixth men left a permanent mark on the game.
While many of his Hall of Fame teammates in the Celtics dynasty overshadowed Ramsey, his immense value should not be underestimated. From the time he shed his unbuttoned warm-up jacket — "I could rip it off as I ran to the scorer's table the second (coach) Red (Auerbach) called my name," he explained in the book "Tall Tales" — he represented an impossible matchup for opponents' reserves.
Know this: No Celtics player came up bigger than Ramsey in the biggest games. In four consecutive postseasons (1957-1960), his field goal percentage and points average exceeded his regular-season output.
Ramsey never was more clutch than in 1959, when he averaged 23.2 points, shot 49.5 percent in the field and led the team in win shares. If not for his team-high 28 points and 11 rebounds in a tense Game 7 of the Eastern Division finals against the Syracuse Nationals, the Celtics would have one fewer championship banner in the rafters.
3. Tyson Chandler
Teams: Chicago Bulls (2000-06), New Orleans/Oklahoma City Hornets (2006-09), Charlotte Bobcats (2009-10), Dallas Mavericks, (2010-11, 2014-15), New York Knicks (2011-14), Phoenix Suns (2015-18), Los Angeles Lakers (2018-19), Houston Rockets (2019-20)
Bottom Line: Tyson Chandler
For nearly two decades, Chandler was among the most consistent rebounders and interior defenders around. His three seasons in New York were the best of his career.
It was there that he earned a Defensive Player of the Year Award and made his lone All-Star Game appearance.
2. Cedric Maxwell
Teams: Boston Celtics (1977-85), Los Angeles Clippers (1986-87), Houston Rockets (1987-88)
Bottom Line: Cedric Maxwell
Maxwell coulda, woulda, shoulda been an all-time Celtics great. Instead, he's widely known as the loose cannon who stole the Most Valuable Player Award from Larry Bird in the 1981 NBA Finals.
Maxwell showed lots of promise in his first two seasons, except that president Red Auerbach already had eyes for Bird, whom he drafted with a year of college eligibility left. Convinced that the golden boy had been signed to take his place, Maxwell never felt the same about the organization again.
In the 1980 draft, Auerbach stole Kevin McHale, another forward. It wasn't long before Maxwell became the odd man odd, but not before he supposedly told his teammates, "Hop on my back, boys, I'll carry you home," before Game 7 of the 1984 NBA Finals.
Guess who had a team-high 24 points, eight rebounds and eight assists in the clincher against the Los Angeles Lakers? After the 1984-85 season, Auerbach asked Bird for input on a potential trade for veteran Bill Walton, the has-been center. Silly question. Bird and McHale were Walton fans from way back. Maxwell was shipped to the fifth-place Los Angeles Clippers, and three years later, at 32, he left the league a bitter man.
So ended the saga of Maxwell, good enough to be an NBA Finals MVP, not good enough to be a Celtic.
1. Manu Ginobili
Teams: San Antonio Spurs
Bottom Line: Manu Ginobili
Many hoops fans outside Texas will tell you their most cherished Ginobili moment took place in 2016 when the Houston Rockets' Ryan Anderson accidentally kicked him in the privates, a low blow that would require testicular surgery. Yep, served Ginobili right for all that moaning and non-stop floppery that cheapened the game, not to mention his reputation.
Funny, but many of those same fans would give their right one to have another El Contusione on their team. Ginobili might have been a bad actor, but the longtime Spur also was an all-time winner. The Argentinian is one of eight players to have an NBA championship ring (four) and Olympic gold medal. He also has a EuroLeague title to his credit.
If not for Ginobili, people like Tim Duncan, Tony Parker and coach Gregg Popovich would haven't have won nearly as much in their careers. He ranks 19th in career postseason Win Shares, ahead of Hall of Famers Elgin Baylor, Julius Erving and Oscar Robertson to name a few.
Whether or not you believe that Ginobili belongs on the list, most can agree on this: He's the best 57th draft pick ev-er.


It was our last day in Paris and we decided that it is a time to see the most beautiful monuments in Paris while enjoying a cruise on boat along the Seine. The cruise departed from Eiffel Tower and lasted an hour, from Eiffel Tower to Louvre Museum and the Cathedral Notre-Dame de Paris, passing many bridges where each one is an unique architectural masterpiece. It was time to add the final touch to our journey and we just relaxed and re-winded our memories when we passed places which we explored earlier. It was cloudy and windy.Teddy was intrigued by sculptures on the bridges and adjacent to them.
When we got back to Eiffel Tower – Teddy asked for "Barbe à papa" (Cotton Candy) and watched with interest how it was gradually growing on a stick until it became a big cotton ball.
Then we headed to Champs-Élysées for shopping. We bought souvenirs and T-shirts for relatives and friends, some clothes for us. Everyone got what wanted. We checked the Disney store there – it has two storeys and so much to offer. The variety of some products there was even better than in Disneyland boutiques. We checked few French brand stores that carry only clothes that made in France. The quality was very good, but they were quite expensive.
It was a lot of souvenir shops and gastronomic boutiques. It was time to go back to hotel and pack our luggage. Early morning we left Paris for Toronto.
2) Our Paris Expectations vs Reality.


We are pleased to announce an addition to our School programme. We now offer weekly support groups for any child who is expressing any difficulty or distress due to a change within their family circumstance or any painful loss. This programme is called RAINBOWS.
When something significant happens in a family, the entire family is affected. Even though death, divorce or other loss issues (such as marital breakdown) appear to be only grown-up problems, they do have a profound effect on the children who are touched by such experiences. If a parent dies, a divorce happens, or some other painful loss occurs, not only do the parents grieve, the children do also. Children may find it extremely difficult to verbalise their feelings of grief because of their age and inexperience.
We have some very sensitive, caring and trained adults who have said "yes" to helping these children put their feelings into words, work through their grief, build a stronger sense of self-esteem, and begin to accept what has taken place in the family.
If your child has experienced the death of a loved family member, friend or sibling, or is struggling through a painful experience, this will be an opportunity for him/her to share with a special someone.
To support our younger children in Reception and Year 1, there is also a group call Sunbeams. This is modelled on the Rainbows programme but uses age appropriate resources. Through SunBeams, children will have the opportunity to grieve the loss in their family and develop appropriate coping mechanisms for life.
Part of the SunBeams programme involves using activities such as puppets, especially designed to help them express their feelings and to learn new coping strategies. The sessions include the following subjects: Self, Divorce and Death, Anger, Family, Belonging, Trust and Coping Tools.


LAS VEGAS–Xytech, a provider of facility management software for the broadcast, media services and video transmission industries is partnering with video cloud provider Blackbird to offer a frame accurate codec player in v9 of Xytech's MediaPulse facility management software system. The companies say the partnership will help streamline the asset management process for users of MediaPulse v9 and provide users with a real-time, accurate browser-based viewer featuring complete access security for all files. Xytech will showcase MediaPulse v9 at the 2019 NAB Show, April 8-11.
The cloud-based Blackbird player will provide MediaPulse users with an efficient, cost-saving viewer through a streaming proxy codec, according to Xytech. By using the integrated player during their workflow, post-production professionals can now frame-accurately play, jog and shuttle backwards and forwards through their video content at multiple speeds, all in the cloud on bandwidth as low as 2 MB per second. Since the content is streamed, deleting the source file instantly renders all stream void for an added a layer of security.
Xytech will be in booth SL4705. Blackbird will be exhibiting on Microsoft's stand, SL6716.
Xytech will use the 2018 NAB Show to launch Dash, a new cloud-based software solution that gives small- to mid-sized media and post facilities robust workflow management capabilities.


text:
Please register or log in to keep reading. No credit card required!
Enjoy unlimited articles at one of our lowest prices ever. You can cancel at any time!
Midwest League
Minor Hockey
Bettendorf News
Cheri Bustos: Trinity Health official takes job in Des Moines
Thomas Geyer
Cheri Bustos, the senior director of corporate communications for Trinity Regional Health Systems for the past six years, is leaving that position to become vice president of corporate communications at Iowa Health System, headquartered in Des Moines, the company announced Wednesday.
Iowa Health System, the largest integrated health system in the state, is the corporate parent of 11 hospitals in large Iowa cities, including three in the Quad-Cities owned by Trinity Regional Health System, which has hospitals in Bettendorf, Rock Island and Moline.
Bustos follows former Trinity president and CEO Bill Leaver, who recently was named the new president and CEO of Iowa Health System.
In her new position, Bustos will oversee marketing and communications for Iowa Health Systems. Her new duties begin Jan. 2.
She said that Leaver has been her boss and mentor for the past six years, and when he asked whether she would be interested in the new position, she took the job.
Bustos also is East Moline Fourth Ward alderman. Although she will work Tuesday through Thursday in Des Moines, she will be able to attend to her City Council duties and make the Monday meetings.
"I'm chairing a revitalization committee, and the effort is to try to bring the downtown back to life," Bustos said. "With cell phones and laptop computers it's easy to stay in touch with people.
"The City of East Moline is still a priority for me."
East Moline Mayor John Thodos said he approached Bustos to run for alderman and is pleased with her contributions to the city.
Bustos is a former editor and writer with the Quad-City Times.
Thomas Geyer can be contacted at
(563) 383-2328 or tgeyer@qctimes.com. Comment on this story at qctimes.com.
Local Crime & Courts
Pleasant Valley schools sued over student video that depicts the beating and execution of a Black slave
A Bettendorf family has filed a lawsuit against the Pleasant Valley School District and its leadership, saying they failed to take action when…
Rail shop in Silvis bought with goal to make it largest steam locomotive museum in the US
Railroading Heritage of Midwest America is buying the former National Railway Equipment shop complex in Silvis to use for the preservation of …
Davenport man arrested on drug and gun charges
A Davenport man wanted in connection with a drug and gun investigation was taken into custody Friday.
Bettendorf doctor announces candidacy for open Iowa Senate district seat
Bettendorf doctor Mary Kathleen Figaro officially announced Tuesday that she is running as a Democrat for the open Iowa Senate District 47 sea…
Man charged with first-degree robbery after pulling a knife on loss-prevention employee over a $199 angle grinder
A Rock Island man who allegedly tried to steal a $199 angle grinder from the Home Depot in Bettendorf on Sunday upped his charge to armed robb…
'There is no time': The toll COVID-19 takes from health care workers
As the number of COVID-19 patients at local hospitals increased, hospitals and healthcare workers face more than care issues. The pandemic is traumatizing those at the front lines.
Blue Grass man charged with sexually abusing teenage girl for almost two years
A Blue Grass man who was recently sentenced to three years probation for growing psilocybin mushrooms has now been charged with two counts of third-degree sexual abuse.
When human remains are found, families 'worry and hope'
When human skeletal remains recently were found in Rock Island County, it became likely a cold case was about to warm up.
7:13+3
'Cool as s**t': Davenport unveils designs for new fire station
The new Davenport fire station planned for North Brady Street will help improve response times, and looks "cool as s**t," said Davenport Mayor Mike Matson.
Latest: No charges to be filed against vape shop employee who shot robbers
Rock Island Police are investigating a shooting incident that left one person dead late Tuesday at Blackhawks Tobacco & Vape, 2733 18th Ave.
QCVarsity.com
Thrifty Nickel
Plus 60
Quad-Cities Calendar
Activate Full Access
© Copyright 2022 The Quad City Times, 500 E 3rd St. Davenport, IA | Terms of Use | Privacy Policy

redpajama_set_name: RedPajamaCommonCrawl


Welcome to Episode 58 of The Good Life Well Lived Podcast and in this episode we are talking about spending the time with the people that matter the most to you.
If you died tomorrow what would you regret? A fairly morbid question but an important one.
In this episode I encourage you to focus your attention on the people that matter the most to you and make sure that you are spending time with those people.
No excuses allowed because I want to make sure that you have no regrets.


text:
Benedetto Vigna: The Man Behind the Chip Behind the Wii
The designer of the MEMS motion sensor in Nintendo's fabulous game tells how he got into micromachining and where he's taking it next
Philip E. Ross
gamingMEMSSTMicroelectronicsWiiaccelerometers
A substantial correction to this article was made on 7 March; to read it, click here.
Benedetto Vigna wields the Wii game's handheld remotes to trade punches with a virtual opponent.RANDI SILBERMAN
Nintendo's Wii is the hottest computer game and arguably the only one that's good for you. Its two wireless remote controls track any movement, encouraging players to engage opponents with a heart-pounding physicality that is already melting fat off overfed children. Yet although detecting motion is critical to the success of the US $250 game, the job depends on $3 sensors the size of shirt buttons.
The supplier of the sensors, STMicroelectronics, got into the business a decade ago in order to squeeze a few more dollars out of an obsolescent chip-making plant. "We wanted something good for it that didn't require deep submicron technology," says Benedetto Vigna, the Italian physicist who developed the sensor.
In 1995, a month after Vigna joined the Geneva-based chip maker, his boss asked him if he was interested in MEMS. Sure, Vigna said, what's MEMS? His boss could only spell out "microelectromechanical systems" and explain that the technology could use outdated photolithographic tools, because it sculpted silicon into things measured in micrometers, not nanometers. It made mechanical the beams, levers, and springs of real, moving machines—the kind a watchmaker would recognize if he had a microscope.
Vigna learned what he could on his own, and a year later he went off to the University of California at Berkeley to study MEMS and do some work-study stints at local companies. Then ST reeled him back, assigned him a staff, and invested heavily in their research. In 2001, three years into the project, Vigna hit on the ideal mass-market product: a chip that could detect motion in three dimensions.
There were already tiny, cheap MEMS devices that could detect motion in two dimensions. That is all that some applications really need—the airbag in a car, for instance, which inflates in the same direction as the collision that it cushions. And there were already big, bulky sensors costing up to $30 000 for airplanes and rockets. Nobody, however, had bothered to fill the gap between the market's high and low ends.
One D, Two D
Motion detection begins with a device called an accelerometer, a cantilever hewn from silicon and teetering between two electrodes. Apply a 1-volt field, and the cantilever's beam will vibrate; accelerate the package, either by pushing it in one dimension or by rotating it, and the beam's tip will trace an ellipse. The eccentricity of the ellipse measures acceleration. Place two such accelerometers at right angles, and you can track acceleration in a plane—add a third, and you can track it in space.
ST's earliest device occupied a cubic inch, a lot smaller than the brick-size gizmos in aircraft, but still way too big for a consumer product. So ST's elves set about shrinking it.
In the photo at right, Vigna [center] shows editors Erico Guizzo [standing] and Philip E. Ross a series of his company's three-dimensional motion sensors. For a close-up of the series, click on the image.
Vigna's team played with various configurations of the basic design—basically, a mass and a spring—and refined the accompanying electronic circuit until it could discern the displacement of fewer than 10 electrons. That way, it could detect very slight motion. The researchers also had to tweak everything so that the sensor would work at both high and low acceleration. "We can measure a flick of the wrist or a big movement of the arm, which wasn't true before at this price level," Vigna says.
To keep costs down, ST found a way to pack the entire mechanical system in plastic rather than in metallic ceramic, as in high-end accelerometers. "Plastic had been considered too weak," Vigna says, "but we designed the whole system to eliminate the package's parasitic effects," that is, its tendency to vibrate along with the sensor.
You might wonder why a device so critical to an expensive product has to sell at $3 rather than $4 or $10. The answer is simple: in the consumer electronics business, every penny counts. Steve Jobs may have convinced millions of people to download songs at 99 cents a pop, but if he'd set the price at $1.25, he'd be known for only two fabulous, world-beating products.
One of the first applications of the 3-D sensor was in laptops, where sensors guard against damage from a fall. In the split second of free fall that comes before the collision with the floor, the sensor tells a controller to park the read/write head safely away from the hard drive.
Another application came in 2003, in a Maytag washing machine that uses a somewhat smaller sensor—14 millimeters by 7 mm by 4 mm—to detect vibrations due to an unbalanced load and to adjust the washer's speed to dampen them.
Another ST product enables the user of a cellphone or a PDA to adjust the display of images or retrieve data from memory by just tilting the device. It sure beats trying to use grown-up fingers to punch commands on baby-size buttons.
Core of the product
Games were already on Vigna's to-do list when he discovered that Nintendo, in Kyoto, was ahead of him. "We met Nintendo in March of 2005—our vision was in line with their vision, and we got married," he says. Two months later, ST delivered a prototype sensor, and 16 months after that, Nintendo launched worldwide sales. Since then, the demand for the game has strained production. IEEE Spectrum could find no Wii boxes at the listed price; to get our test version, we paid a $100 markup to an enterprising reseller on eBay.
Vigna says the Wii has been the biggest application of all "because the MEMS chip is the core of the product." (To be sure, the motion detector doesn't do the whole job by itself; an infrared system helps, by setting the player's initial position.)
In recognition of his work, ST recently made Vigna the general manager of its MEMS product division. That means he has to plan for the long term.
First he wants to make the sensor even smaller, even cheaper, even tougher. "I want it to fit in all kinds of places—shoes and textiles, for instance, where it might be useful for medical monitoring," he says.
"Then I want to make a three-dimensional gyroscope, to measure rotation around three different axes. Today, such products are quite big, a cube 10 centimeters on a side. We want to do this in less than a 30-millimeter cube, to serve as an image stabilizer in cameras and to track a person's position in the intervals when he can't get a GPS signal."
Better still, he adds, would be to throw in a magnetic detector, freeing the navigator from GPS altogether. It would be yet another marvel from Lilliput—the smallest compass ever sold.
SensorsNewsTypeTopic
Entangling Quantum Sensors Can Triple Accuracy
SensorsTopicTypeNewsTransportation
Crowdsourcing Bridge Inspection
Artificial IntelligenceTopicTypeNewsTelecommunications
When Gamers Get Nasty
This article is for IEEE members only. Join IEEE to access our full archive.
Join the world's largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum's articles, podcasts, and special reports. Learn more →
If you're already an IEEE member, please sign in to continue reading.
BECOME A MEMBERSIGN IN
Get unlimited access to IEEE Spectrum content
Follow your favorite topics to create a personalized feed of IEEE Spectrum content
Save Spectrum articles to read later
Network with other technology professionals
Establish a professional profile
Create a group to share and collaborate on projects
Discover IEEE events and activities
Join and participate in discussions
Consumer ElectronicsTopicMagazineTypeFeatureFebruary 2023
The Inner Beauty of Basic Electronics
Open Circuits showcases the surprising complexity of passive components
Eric Schlaepfer
Windell H. Oskay
All photos by Eric Schlaepfer & Windell H. Oskay
Eric Schlaepfer was trying to fix a broken piece of test equipment when he came across the cause of the problem—a troubled tantalum capacitor. The component had somehow shorted out, and he wanted to know why. So he polished it down for a look inside. He never found the source of the short, but he and his collaborator, Windell H. Oskay, discovered something even better: a breathtaking hidden world inside electronics. What followed were hours and hours of polishing, cleaning, and photography that resulted in Open Circuits: The Inner Beauty of Electronic Components (No Starch Press, 2022), an excerpt of which follows. As the authors write, everything about these components is deliberately designed to meet specific technical needs, but that design leads to "accidental beauty: the emergent aesthetics of things you were never expected to see."
From a book that spans the wide world of electronics, what we at IEEE Spectrum found surprisingly compelling were the insides of things we don't spend much time thinking about, passive components. Transistors, LEDs, and other semiconductors may be where the action is, but the simple physics of resistors, capacitors, and inductors have their own sort of splendor.
High-Stability Film Resistor
This high-stability film resistor, about 4 millimeters in diameter, is made in much the same way as its inexpensive carbon-film cousin, but with exacting precision. A ceramic rod is coated with a fine layer of resistive film (thin metal, metal oxide, or carbon) and then a perfectly uniform helical groove is machined into the film.
Instead of coating the resistor with an epoxy, it's hermetically sealed in a lustrous little glass envelope. This makes the resistor more robust, ideal for specialized cases such as precision reference instrumentation, where long-term stability of the resistor is critical. The glass envelope provides better isolation against moisture and other environmental changes than standard coatings like epoxy.
15-Turn Trimmer Potentiometer
It takes 15 rotations of an adjustment screw to move a 15-turn trimmer potentiometer from one end of its resistive range to the other. Circuits that need to be adjusted with fine resolution control use this type of trimmer pot instead of the single-turn variety.
The resistive element in this trimmer is a strip of cermet—a composite of ceramic and metal—silk-screened on a white ceramic substrate. Screen-printed metal links each end of the strip to the connecting wires. It's a flattened, linear version of the horseshoe-shaped resistive element in single-turn trimmers.
Turning the adjustment screw moves a plastic slider along a track. The wiper is a spring finger, a spring-loaded metal contact, attached to the slider. It makes contact between a metal strip and the selected point on the strip of resistive film.
Ceramic Disc Capacitor
Capacitors are fundamental electronic components that store energy in the form of static electricity. They're used in countless ways, including for bulk energy storage, to smooth out electronic signals, and as computer memory cells. The simplest capacitor consists of two parallel metal plates with a gap between them, but capacitors can take many forms so long as there are two conductive surfaces, called electrodes, separated by an insulator.
A ceramic disc capacitor is a low-cost capacitor that is frequently found in appliances and toys. Its insulator is a ceramic disc, and its two parallel plates are extremely thin metal coatings that are evaporated or sputtered onto the disc's outer surfaces. Connecting wires are attached using solder, and the whole assembly is dipped into a porous coating material that dries hard and protects the capacitor from damage.
Film Capacitor
Film capacitors are frequently found in high-quality audio equipment, such as headphone amplifiers, record players, graphic equalizers, and radio tuners. Their key feature is that the dielectric material is a plastic film, such as polyester or polypropylene.
The metal electrodes of this film capacitor are vacuum-deposited on the surfaces of long strips of plastic film. After the leads are attached, the films are rolled up and dipped into an epoxy that binds the assembly together. Then the completed assembly is dipped in a tough outer coating and marked with its value.
Other types of film capacitors are made by stacking flat layers of metallized plastic film, rather than rolling up layers of film.
Dipped Tantalum Capacitor
At the core of this capacitor is a porous pellet of tantalum metal. The pellet is made from tantalum powder and sintered, or compressed at a high temperature, into a dense, spongelike solid.
Just like a kitchen sponge, the resulting pellet has a high surface area per unit volume. The pellet is then anodized, creating an insulating oxide layer with an equally high surface area. This process packs a lot of capacitance into a compact device, using spongelike geometry rather than the stacked or rolled layers that most other capacitors use.
The device's positive terminal, or anode, is connected directly to the tantalum metal. The negative terminal, or cathode, is formed by a thin layer of conductive manganese dioxide coating the pellet.
Axial Inductor
Inductors are fundamental electronic components that store energy in the form of a magnetic field. They're used, for example, in some types of power supplies to convert between voltages by alternately storing and releasing energy. This energy-efficient design helps maximize the battery life of cellphones and other portable electronics.
Inductors typically consist of a coil of insulated wire wrapped around a core of magnetic material like iron or ferrite, a ceramic filled with iron oxide. Current flowing around the core produces a magnetic field that acts as a sort of flywheel for current, smoothing out changes in the current as it flows through the inductor.
This axial inductor has a number of turns of varnished copper wire wrapped around a ferrite form and soldered to copper leads on its two ends. It has several layers of protection: a clear varnish over the windings, a light-green coating around the solder joints, and a striking green outer coating to protect the whole component and provide a surface for the colorful stripes that indicate its inductance value.
Power Supply Transformer
This transformer has multiple sets of windings and is used in a power supply to create multiple output AC voltages from a single AC input such as a wall outlet.
The small wires nearer the center are "high impedance" turns of magnet wire. These windings carry a higher voltage but a lower current. They're protected by several layers of tape, a copper-foil electrostatic shield, and more tape.
The outer "low impedance" windings are made with thicker insulated wire and fewer turns. They handle a lower voltage but a higher current.
All of the windings are wrapped around a black plastic bobbin. Two pieces of ferrite ceramic are bonded together to form the magnetic core at the heart of the transformer.
This article appears in the February 2023 print issue.
Dell Tried to Hide Bad Capacitors Problem 2003-2005 ›
Hands On - IEEE Spectrum ›
Watch: Laser Origami Makes Inductors ›
Open Circuits: The Inner Beauty of Electronic Components: Oskay ... ›
Open Circuits | No Starch Press ›
Open Circuits ›


Anyone with the correct digital camera equipment, and the mandatory abilities can set up a house business, advertising and marketing pictures. Photography prospects (current and potential) want to know that we can, and can, produce the best quality photography work for them. Rights Managed Non-Exclusive – An organization buys time-restricted rights to a picture, but other companies are also free to license the identical picture.
Each time I have been impressed by their professionalism and the quality of work that they supply. Prime quality service by delivering the completed merchandise to the shopper. For those who're able to launch your personal photography enterprise, studio or freelance work, then you definitely want a outstanding and artistic name for your corporation.
My model as a photographer is a hybrid between photo-journalism, art photography and portrait images. Other resort images services can take months to get to your resort whereas RTV could have your lodge pictures order completed inside two to three weeks.
A enterprise insurance coverage may be custom-made to help cowl a images business. Great images can be utilized to painting your brand as professional, thrilling, or even cheeky. It was a pleasure working together with your photographer, she was extremely professional, bubbly and a pleasure to work with.
Our work covers many various kinds of photographic assignment, ranging from individual and group portraits, firm occasions (services or products launches, firm parties, conferences), annual report and placement photographs, and company video manufacturing.


E (Europa) was sold for 38p on day of issue. Background colors are printed with Iriodin ink, giving the stamp a three dimensional appearance.
Prince William, the first child of the Prince and Princess of Wales, was born atSt Mary's Hospital, London, on 21 June 1982. A week later on 28 June his name was announced by Buckingham Palace: William Arthur Philip Louis. He was baptised in the Music Room of Buckingham Palace on 4 August 1982 (the 82nd birthday of his paternal great-grandmother, Queen Elizabeth The Queen Mother) by the Archbishop of Canterbury, Robert Runcie.
He was the first child born to a prince and princess of Wales since Prince John in 1905.
William's first public appearance was on 1 March 1991 (Saint David's Day), during an official visit of his parents to Cardiff, Wales. After arriving by aeroplane, William was taken to Llandaff Cathedral where he signed the visitors' book, thereby demonstrating that he was left-handed.
At the age of 21, Prince William was appointed as a Counsellor of State, and began his royal duties by first serving in that capacity when the Queen was in Nigeria to attend the Commonwealth Heads of Government Meeting 2003. For his 21st birthday, William accompanied his father on a royal tour of Wales, visiting the Anglesey Food Fair and opening a centre for the homeless in Newport.


The most expensive part of your laptop is the actual screen itself, so to replace it can be very costly, even cleaning incorrectly will cause damage leaving big white marks, and the only way to fix it is by replacing it. Here are a few hints on how to clean your screen without damaging it.Buy a good quality LCD/LED screen cleaner, and always spray the cloth with the solution never spray the screen directly, wipe very gently in one direction top to bottom with the cloth provided which is normally a very soft lint free cloth similar to that of what you receive when you purchase a pair of glasses.On a budget, you can make your own solution to clean your computer, laptop, iPad or television screen.Make sure the screen monitor is off, you will be able to identify the dirty areas of the screen easier, all screens have a tendency to attract dust easily and you should never touch the screen and leave dirty finger prints this can cause problems to the screen also.Always be very gentle and never put even the slightest pressure on the screen with your h and. Find a piece of very soft cloth – preferably very soft cotton, eye glass lint free cloths is great to use, don't use tissues or tissue paper, toilet paper or even towels as the fibers from these will scratch the screen permanently.You can make your own solution for cleaning LED/LCD monitors, iPad, televisions etc, you will need de-ionized or distilled water and vinegar or iso-propyl alcohol – Don't use tap water or mineral water as they could leave white marks on the screen because of the dissolved salts.Dilute the vinegar or IPA with distilled water [1:1] spray directly the solution onto the soft cloth so it is moist not saturated. Never spray it directly onto the screen. Gently move the moistened cloth in just one direction – from top to bottom. Make sure the screen is completely dry before closing it down, otherwise the moisture will cause damage to the screen.


Templeton Rye, the Iowa-based whiskey manufacturer, has broken ground on its new $26 million distillery building.
The new 99,000-square foot complex includes a 34,500-square foot distillery as well as a museum and barrel-aging warehouse.
It's expected to be completed in the fall of 2018.
The new facility may muzzle critics who claim Templeton had been misrepresenting its liquor by labeling it as "small-batch rye" and "made in Iowa," when it actually was made from a stock whiskey distilled by MGP Ingredients in Indiana.
Complaints culminated in a series of lawsuits over the whiskey's authenticity, which the company settled in 2015 with promises to change its marketing materials and bottle labeling.
More:Will Templeton Rye's Iowa move change its taste?
Currently, Templeton Rye has six full-time and about 10 part-time employees, the company said.
The distillery project will add 27 jobs and will be able to produce 500,000 proof gallons of rye whiskey annually. The aging warehouse will be able to store 40,000 barrels.
— Kevin Hardy contributed to this report.


text:
\section{Introduction}
We consider the finite sum optimization problem:
\beq \label{eq:opt}
\min_{ \prm \in \RR^d }~ F(\prm)
~~\text{where}~~F(\prm) \eqdef \frac{1}{N} \sum_{i=1}^N f_i (\prm)  \eqs.
\eeq
Each component function
$f_i : \RR^d \rightarrow \RR$ is assumed to be convex and twice continuously differentiable with Lipschitz continuous gradients,
while the sum function $F(\prm)$ is assumed to be strongly convex.
Such problem customarily arises in several multi-agent systems applications \cite{nedic2009distributed}, e.g., communication networks~\cite{lin2006tutorial}, sensor networks~\cite{freris2010fundamentals}, and distributed learning~\cite{boyd2011distributed}.
In a distributed setting, $f_i (\cdot)$ is the \emph{private}
function known only to agent $i$; for example, in a learning task such as empirical risk minimization (ERM)~\cite{vapnik1999overview}, this corresponds to a subset of data gathered by agent $i$.

This paper
develops an efficient distributed algorithm for \eqref{eq:opt} under a general communication topology. In particular, we adopt
a stochastic asynchronous setting that captures activation of agents in networked decision and control applications.
To this end, a large body of prior work has focused on developing distributed algorithms based on the average consensus protocol \cite{Tsitsiklis1984}, e.g.,
as introduced by {Nedi\'{c}} \emph{et al.} \cite{nedic2001distributed}.
However, the convergence rate for this class of methods is limited by
the diameter of the communication graph \cite{scaman2017optimal}.
To develop an algorithm that is robust to the graph structure,
we consider here an alternative pathway that is
akin to that described in \cite{johansson2009randomized,ram2009incremental},
for utilizing stochastic algorithms for finite sum minimization
via pairwise exchange of information on a graph \cite{nedic2009distributed,RK}.
Inspired by the classical work of Robbins and Monro~\cite{robbins1951stochastic}, stochastic algorithms have been actively pursued for machine learning operations over the last decade, with notable recent examples
enlisting stochastic average gradient (SAG)\cite{SAG} and its unbiased
variant (SAGA)~\cite{SAGA}, and stochastic variance reduction gradient (SVRG)~\cite{SVRG}. All these methods (including the one we propose here)
are \emph{incremental}~\cite{nedic2001incremental} and operate using the following key steps, at each iteration:
a) a \emph{random} component function is selected, and its gradient is evaluated at the current iterate, b) the obtained gradient value is aggregated into the memory, and c) a new iterate is obtained using the aggregated gradient. Additionally, they require storing the gradient values to allow for an asynchronous implementation.

Inspired by \cite{johansson2009randomized,ram2009incremental},
we make an observation that the \emph{random selection-aggregate-update} paradigm can be realized
in a fully distributed fashion via a random walk on a graph, where a selected agent forwards information to one of its neighbors, \ie  essentially implementing a Markov-driven token passing protocol;
cf.~Section~\ref{sec:dist}.
Most notably, while
stochastic algorithms such as SAG and SAGA are known
to
converge linearly for strongly convex problems~\cite{SAG,SAGA},
their rates depend on the number of agents $N$.
For example, SAGA requires
${\cal O}( ( N + \kappa(F) ) \log (1/\epsilon) )$ steps in expectation
to reach an $\epsilon$-optimal solution, where $\kappa(F)$
denotes the condition number for the sum function $F(\cdot)$.

In our recent work, we proposed the \emph{Curvature-aided Incremental Aggregated Gradient} (CIAG)~\cite{CIAG} method, which utilizes curvature (\ie Hessian) information to acquire accelerated convergence. Under the asynchronous model of computation~\cite{bertsekas1989parallel} (where each component function is selected by the algorithm infinitely often, with a uniformly bounded delay), CIAG was shown to converge linearly at a rate that is strictly faster than SAGA;
see also~\cite{gower2017tracking} for a similar technique to variance
reduction in stochastic gradient methods.
Nevertheless, CIAG is not directly amenable to a distributed implementation,
where it would require message-passing among agents along a closed spanning walk following
a \emph{fixed} sequence. This is typically
restrictive,
especially in distributed wireless sensor networks where time synchronization is challenging~\cite{freris2011fundamental} and it is desirable that agents
are activated at random time instants
for the sake of battery lifetimes.

This paper
adopts a stochastic setting where the component
functions are selected at random, \ie it allows for a Markov-driven distributed
implementation on a general topology.
Moreover, the proposed \emph{Stochastic Unbiased Curvature-aided Gradient}
(SUCAG) method introduces an \emph{unbiased}
curvature-aided gradient estimator.
We analyze the convergence of SUCAG for strongly convex
problems and show that it converges linearly \emph{with high probability} (w.h.p.), for a sufficiently small stepsize selection.
Besides,
for an initialization sufficiently close to optimality, the algorithm requires ${\cal O}( \kappa(F) \log (1 / \epsilon))$
steps to reach an $\epsilon$-optimal solution,
\emph{robust to the graph structure or network size}.
This can be beneficial in the prelude of large-scale cyberphysical systems \cite{CPS}
that feature millions of agents, for example in real-time learning from streaming data~\cite{RCS,RCS2}.
\vspace{.1cm}

\textbf{Notation}.
We denote vectors and matrices using boldface lower-case letters and upper-case letters,
respectively.
We use the notation $(x)_+:=\max\{0,x\}$ for the positive part.
We use the standard Bachmann-Landau notation: for non-negative functions $f,g,h$ we write $f(t) = {\cal O}( g(t) )$ (\resp $f(t) = \Omega(h(t))$) when there exists a positive constant $c$ (\resp $C$) such that $f(t) \leq c g(t)$ ($f(t) \geq C h(t)$).
We use $\| \cdot \|$ for the standard Euclidean norm.

\subsection{Overview of Gradient Tracking Techniques} \label{sec:ciag}
Before we introduce the proposed SUCAG algorithm, we briefly
review the curvature-aided gradient tracking technique. In particular, we cast it as a specific variance reduction method
within the stochastic gradient paradigm.

When the number of components $N$ is large,
and especially in a distributed setting,
the total gradient $\grd F$ is costly to evaluate,
whence a stochastic gradient (SG) scheme is
usually sought to tackle \eqref{eq:opt} in a tractable manner.
To this end, a SG scheme
can be described as follows: at iteration $k \in \NN$,
a \emph{stochastic gradient surrogate} ${\bm g}^k$ is obtained, satisfying
$\EE [ {\bm g}^k ] = \grd F( \prm^k ),~{\rm Var}( {\bm g}^k ) \leq \sigma^2$,
\ie an unbiased estimator for $\grd F(\prm^k)$ with
finite error variance. A common implementation
selects one of the component functions uniformly at random, \ie
the $i_k-$th function (where $i_k \in [N]$)
at iteration $k$, and evaluates the gradient $\grd f_{i_k}(\prm^k).$

The SG method~\cite{robbins1951stochastic} then solves \eqref{eq:opt} using the recursion
$\prm^{k+1}
= \prm^k - \gamma_k {\bm g}_{\sf SG}^k$
with ${\bm g}^k_{{\sf SG}}\eqdef\grd f_{i_k}(\prm^k)$, where
$\gamma_k >0$ is the stepsize at the $k-$th iteration:  when $\{\gamma_k\}$ satisfies
$\sum_k \gamma_k = \infty, \sum_k \gamma_k^2 < \infty$,
the SG method converges to an optimal solution of \eqref{eq:opt}
almost surely (a.s.) \cite{robbins1951stochastic}.
However, albeit its simplicity,
a major drawback of SG remains that
the (expected) convergence rate is \emph{sublinear}, \ie
$\EE [\| \prm^\star - \prm^k \| ]= {\cal O}(1/k)$ (where $\prm^\star$
denotes the optimal solution to \eqref{eq:opt})
when the objective function is strongly convex \cite{moulines2011non}.

On the other hand, one may adopt an \emph{incremental} approach to the problem.
In this purview, a reasonable remedy to the high cost of evaluating the total gradient is to \emph{aggregate} previous gradient component evaluations, \ie approximate
$\grd f_j (\prm^k ) \approx \grd f_j ( \prm^{\tau_{j}^{k}} )$
where $\tau_{j}^{k}$ denotes the iteration count when the $j-$th gradient was last evaluated (see \eqref{eq:tau} for
the precise definition of $\tau_i^k$). This is precisely the mechanism of SAGA~\cite{SAGA} that selects:\vspace{-.2cm}
\beq\label{eq:saga} \notag
{\bm g}^k_{\sf SAGA}\eqdef \left[\grd f_{i_k}(\prm^k) - \grd f_{i_k}(\prm^{\tau_{i_k}^{k-1}})\right] + \frac{1}{N}\sum_{j=1}^N\grd f_{j}(\prm^{\tau_j^{k-1}}).
\vspace{-.1cm}
\eeq
Besides, SAG is different only in that it multiplies the first term above by $\frac{1}{N}$ and is, therefore, biased.
When each component function $f_j$ is smooth, the approximation error, \ie the \emph{variance} of
the total gradient estimator,
is bounded by
${\cal O} ( \sum_{i=1}^N \| \prm^{\tau_i^{k}} - \prm^\star \|)$.
Thanks to the \emph{variance reduction} property, the SAGA/SAG  methods
are shown to converge at a linear rate~\cite{SAGA,SAG} and are therefore faster than SG, at the cost of storing the past gradient component values. It is worthwhile observing that in a multi-agent setting, past values can be stored locally, and aggregated in a recursive manner via message passing. The crucial implementation detail in our algorithm is that aggregation can be performed via simple \emph{gossiping}, \ie message exchange between an agent and only one of its neighbors.

Under an additional smoothness condition (of Lipschitz-continuous component Hessians),
our recent work \cite{CIAG} considered
a first-order Taylor approximation to the
total gradient
 in order to further reduce the estimation variance: \vspace{-.1cm
\beq \label{eq:ciag}
{\bm g}^k_{\sf CIAG}\eqdef \frac{1}{N}\sum_{j=1}^N \left[\grd f_j( \prm^{\tau_j^{k}} ) + \grd^2 f_j( \prm^{\tau_j^{k}} )
( \prm^k -  \prm^{\tau_j^{k}} )\right],\notag\vspace{-.1cm}
\eeq
A distinctive attribute is that the approximation error for ${\bm g}^k_{\sf CIAG}$
to the total gradient
above can be bounded
by ${\cal O}( \sum_{j=1}^N\| \prm^k - \prm^{\tau_j^k} \|^2 )$
in light of~\cite[Lemma 1.2.4]{nesterov_lectures},
\ie the bound features \emph{squared norm}
of the
distance to optimality; specifically, when $\| \prm^k - \prm^{\tau_j^k} \|^2 < 1$ (as is the case for large enough $k$), this bound allows a tighter convergence analysis as compared to SAG/SAGA.
In brief, it was shown in \cite{CIAG}
that CIAG method exhibits a faster linear convergence rate than SAG \emph{by a factor of $N$}, under the restriction that the delays in gradient/Hessian evaluation are
bounded (\ie $| \tau_i^k - k | \leq K < \infty$, e.g., when $i_k$
is chosen as $i_k = ( k ~{\rm mod}~ N ) + 1$). Note, however, that in a stochastic setup CIAG is biased, which motivates the development of an unbiased variant in this paper.

\section{The SUCAG method}
We first introduce SUCAG method as a stochastic gradient algorithm for finite sum optimization \eqref{eq:opt}, and
then proceed to present
protocols for implementing it in a distributed setting.

We adhere to the stochastic optimization setting:
at iteration $k$, the algorithm selects
the $i_k-$th component function $f_{i_k} (\cdot)$, where $i_k \in \{1,...,N\}$, and evaluates its gradient/Hessian at the current iterate. At first, we assume that the selection mechanism is independent identically distributed (i.i.d.), where a
random integer is chosen \emph{uniformly at random}, \ie $P(i_k = j) = 1/N$ for all $j =1,...,N$.
We define the random variable $\tau_j^k$ as the iteration number where the $j-$th function was last accessed
(including iteration $k$ itself):
\beq \label{eq:tau}
\tau_j^k := \max\{\ell \geq 0 ~:~ i_\ell = j,~\ell \leq k \} \eqs.
\eeq
Note that $\tau_{i_k}^k = k$ and $\tau_j^k = \tau_j^{k-1}$ for all $j \neq i_k$.

Inspired by the curvature-aided gradient tracking technique~\cite{CIAG},
the SUCAG method adopts the update rule:
\beq \label{eq:sucag}
\prm^{k+1} = \prm^k - \gamma {\bm g}_{\sf SUCAG}^k,~k \geq 0 \eqs,
\eeq
where $\gamma > 0$ is a fixed step size
and ${\bm g}_{\sf SUCAG}^k$ is a curvature-aided approximation
on the actual gradient
$\grd F( \prm^k )$. The approximation is given as\footnote{Note that CIAG pre-multiplies the first term with $\frac{1}{N}$, in the same way that SAG does for SAGA.}:
\beq \label{eq:ucag_g}
\begin{split}
& \textstyle {\bm g}_{\sf SUCAG}^k \eqdef \left[ \grd f_{i_k}(\prm^k) - {\bm g}_{i_k}^k (\prm^k) \right] + \frac{1}{N} \sum_{i=1}^N {\bm g}_i^k (\prm^k)  \eqs,
\end{split}
\eeq
where
\beq \label{eq:sucag_e}
{\bm g}_i^{k} (\prm) \eqdef \grd f_i (\prm^{\tau_i^{k-1}}) + \grd^2 f_i (\prm^{\tau_i^{k-1}}) \big( \prm - \prm^{\tau_i^{k-1}} \big) \eqs.
\eeq
A key property of ${\bm g}_{\sf SUCAG}^k$ is that it is an
\emph{unbiased} estimator for $\grd F(\prm^k)$.
To see this, define the filtration ${\cal F}_k$ as the collection of random variables
(the chosen indices $\{i_k\}$)
realized prior to the update of $\prm^{k+1}$, \ie ${\cal F}_k\eqdef \sigma(\{i_{\ell}:\ell = 0,1,\hdots,k-1\})$;
note that this does not include the newly selected index  $i_k$.
The conditional expectation is given by:
\beq \label{eq:unbiased} \begin{split}
\EE_k[ {\bm g}_{\sf SUCAG}^k ] &   = \textstyle
\frac{1}{N} \sum_{i=1}^N \big( {\bm g}_i^k (\prm^k) + \grd f_i(\prm^k) -
{\bm g}_i^k( \prm^k ) \big)  \\
& = \grd F( \prm^k ) \eqs,
\end{split}
\eeq
where $\EE_k[\cdot] \eqdef \EE[ \cdot | {\cal F}_k ]$, and
therefore SUCAG is unbiased.
Note that SUCAG is a natural extension of SAGA to incorporate curvature information.

\algsetup{indent=0.8em}
\begin{algorithm}[t]
\caption{SUCAG method}\label{alg:sucag}
  \begin{algorithmic}[1]
  \STATE \textbf{Input}: Initial point $\prm^0 \in \RR^d$
  \STATE Set counter variables $\tau_i^0 \eqdef -1$ for all $i$.
  \STATE Set $\grd f_i( \prm^{-1} ) = {\bm 0}$ and $\grd^2 f_i ( \prm^{-1} ) = {\bm 0}$ for all $i$.
  \FOR {$k=0,1,2,\hdots, K$}
  \STATE Choose $i_k \in \{1,\hdots,N\}$ uniformly at random.
  \STATE Compute ${\bm g}_{\sf SUCAG}^k$ using \eqref{eq:ucag_g},
  \eqref{eq:sucag_e}.
   \STATE
   \textbf{Update:}
   \[
   \prm^{k+1} = \prm^k - \gamma {\bm g}^k_{\sf SUCAG} \eqs. \vspace{-.5cm}
   \]
   \STATE Update counter variables: $\tau_{i_k}^k \leftarrow k$,
   and $\tau_j^k \leftarrow \tau_j^{k-1}$ for all $j \neq i_k$.
\ENDFOR
\STATE \textbf{Return}
$\prm^{K+1}$.
  \end{algorithmic}
\end{algorithm}
While our main focus is on applying SUCAG in a distributed setting, we note that one can also implement the
SUCAG method as a centralized incremental method just as in CIAG~\cite{CIAG}. In this case, the system stores ${\cal O}(Nd)$ real numbers in the memory for the
previous iterates and the per-iteration
complexity is ${\cal O}(d^2)$, cf.~\cite{CIAG}.

\subsection{Distributed Implementation}
\label{sec:dist}
Consider a \emph{connected} undirected
graph $G = (V,E)$ where $V = \{1,\hdots,N\}$
is the set of $N$ agents and $E \subseteq V \times V$ is the edge set that prescribes the
agent-to-agent communication pairs.
The neighborhood set of $i$ is $\mathcal{N}_i\eqdef \{j: (i,j)\in E\}$.

It is easy to verify that:
\beq \label{eq:sucag_a}
{\bm g}_{\sf SUCAG}^k = \left[ \grd f_{i_k}(\prm^k) - {\bm g}_{i_k}^k (\prm^k) \right]+ {\bm b}^{k-1} + {\bm H}^{k-1} \prm^k,
\eeq
where
\beq \notag
\begin{split}
& \textstyle {\bm b}^{k-1} \eqdef \frac{1}{N} \sum_{i=1}^N \big( \grd f_i( \prm^{\tau_i^{k-1}} ) -
\grd^2 f_i( \prm^{\tau_i^{k-1}} ) \prm^{\tau_i^{k-1}} \big) \eqs, \\
& \textstyle {\bm H}^{k-1} \eqdef \frac{1}{N} \sum_{i=1}^N  \grd^2 f_i( \prm^{\tau_i^{k-1}} ) \eqs.
\end{split}
\eeq
These variables can be computed recursively as follows:
\beq \label{eq:inc_upd}
\begin{split}
& \textstyle {\bm b}^{k} \eqdef {\bm b}^{k-1} + \frac{1}{N} \left( \grd f_{i_k}( \prm^k ) - \grd f_{i_k}( \prm^{\tau_{i_k}^{k-1}})\right) \\
& \textstyle \hspace{1.35cm} \frac{1}{N}\left(\grd^2 f_i(  \prm^{\tau_{i_k}^{k-1}} )  \prm^{\tau_{i_k}^{k-1}}- \grd^2 f_i( \prm^k ) \prm^k \right)\\
& \textstyle {\bm H}^{k} \eqdef {\bm H}^{k-1} + \frac{1}{N} \big(
\grd^2 f_{i_k} ( \prm^k ) -  \grd^2 f_{i_k} ( \prm^{\tau_{i_k}^{k-1}} )
\big) \eqs.
\end{split}
\eeq
Leveraging these recursions, we can obtain distributed protocols for implementing SUCAG on $G$,
via transmitting the aggregated (previous) gradients and Hessians. To this end, a key aspect is that the required updates for SUCAG [cf.~\eqref{eq:sucag}, \eqref{eq:sucag_a}, \eqref{eq:inc_upd}] are \emph{locally computable} by an activated agent $i_k$, given the latest estimate $\prm^k$ and aggregate gradient/Hessian information ${\bm b}^{k-1}, {\bm H}^{k-1}$.

First, for the special case when $G$ is a \emph{star graph}, a simple protocol can be implemented by using the hub node as a coordinator to store the tuple $\{{\bm b}^{k-1}, {\bm H}^{k-1}, \prm^k\}$ and, at each iteration: \emph{(i)} the coordinator activates an agent independently and uniformly at random and passes along this tuple, \emph{(ii)} the agent performs the required updates [cf.~\eqref{eq:inc_upd},\eqref{eq:sucag_a},\eqref{eq:sucag}] and \emph{(iii)} transmits back the updated values $\{{\bm b}^k, {\bm H}^k, \prm^{k+1}\}$.
Such protocol is ideal for cloud computing services.

\begin{figure}[t]
\centering
\ifplainver
{\sf \resizebox{.6\linewidth}{!}{\input{./Tikz/SUCAG.tikz.tex}}}\vspace{-.1cm}
\else
{\sf \resizebox{.9\linewidth}{!}{\input{./Tikz/SUCAG.tikz.tex}}}\vspace{-.1cm}
\fi
\caption{Distributed implementation of SUCAG; the agent sequence $\{ i_k \}_{k \geq 1}$ corresponds to a random walk on the graph $G$.}\vspace{-.4cm} \label{fig:sucag}
\end{figure}

A crucial point of our analysis (cf.~Theorem~\ref{thm:main})
reveals that the requirement of independent and uniform agent activation \emph{can be relaxed}.
In fact, it is adopted primarily for demonstration purposes
(to ensure unbiasedness, $\EE_k[ {\bm g}^k_{\sf SUCAG} ] = \grd F(\prm^k)$)
so as to portray our scheme within the popular stochastic variance
reduction framework.
This important attribute allows for distributed computations along a random walk on
a connected graph $G$ as follows:
at iteration $k$, activated agent $i_k$ performs  the update~\eqref{eq:sucag} and \emph{(i)} stores $\prm^{k}$\footnote{In light of storage/computation trade-off, it is also possible for agent $i_k$ to store $\{\prm^{k+1},\grd f_{i_k}(\prm^k),\grd^2f_{i_k}(\prm^k)\}$ in order to avoid re-evaluation at the next activation instance.},
\emph{(ii)} chooses one of its neighbors $i_{k+1} \in {\cal N}_{i_k}$ at random
as the next active agent, and \emph{(iii)} transmits the updated $\{{\bm b}^k, {\bm H}^k, \prm^{k+1}\}$
to agent $i_{k+1}$. This process is illustrated in
Fig.~\ref{fig:sucag}.

We remark that the above line of reasoning directly applies to \emph{any} stochastic algorithm with the \emph{random selection-aggregate-update} paradigm, e.g., SG, SAG and SAGA; see \cite{johansson2009randomized,ram2009incremental}.

\section{Convergence Analysis}
This section establishes
the linear convergence of the SUCAG method
with high probability.
We begin by stating the required assumptions
on the objective function of \eqref{eq:opt}:\vspace{-.2cm}
\begin{Assumption}\label{ass:cts_h}
Each component function $f_i (\prm)$ has $L_{H,i}$-Lipschitz continuous Hessian. In other words, for any $i\in \{1,...,N\}$, there exists $L_{H,i}>0$, such that for all $\prm', \prm \in \RR^d$ \vspace{-.1cm}
\beq \begin{split}
& \| \grd^2 f_i ( \prm ) - \grd^2 f_i (\prm' ) \| \leq L_{H,i} \| \prm - \prm' \| \eqs.
\end{split} \vspace{-.1cm}
\eeq
We define $\bar{L}_H \eqdef \max_{i\in \{1,...,N\}} L_{H,i}$.
\end{Assumption}\vspace{-.4cm}
\begin{Assumption}\label{ass:cts}
The gradient of sum function $F(\prm)$ is $L$-Lipschitz continuous,
\ie
there exists $L>0$, such that for all $\prm', \prm \in \RR^d$, \vspace{-.4cm}
\beq \begin{split}
& \| \grd F ( \prm ) - \grd F (\prm' ) \| \leq L \| \prm - \prm' \| \eqs.
\end{split}
\eeq
\end{Assumption}\vspace{-.4cm}
\begin{Assumption}\label{ass:strcvx}
The sum function $F (\prm)$ is $\mu$-strongly convex, $\mu > 0$, \ie for all $\prm', \prm \in \RR^d$,
\beq
F( \prm' ) \geq F( \prm ) + \langle \grd F(\prm), \prm' - \prm \rangle + \frac{\mu}{2} \| \prm' - \prm \|^2 \eqs.
\eeq
\end{Assumption}
Under A\ref{ass:strcvx}, a unique optimal solution
to problem~\eqref{eq:opt} exists and it is denoted by $\prm^\star$.
We denote the condition number for $F(\prm)$
by $\kappa(F) \eqdef L / \mu$.
Note that the above are standard assumptions that can be satisfied in many
applications; for example, $\ell_2-$regularized logistic regression~\cite{vapnik1999overview}, and quadratic programming such as (overdetermined) least-squares and Model Predictive Control (MPC)~\cite{morari1999model} applications.

We also state the following assumption on the delays in the SUCAG method:\vspace{-.2cm}
\begin{Assumption} \label{ass:delay} \label{ass:delay2}
	For all $k \geq 1$, it holds that $|k - \tau_i^{k-1}| \leq m_k$ for all $i$, where $\{m_k\}_{k \geq 1}$ is non-decreasing with $m_1\ge 1$. Moreover, it holds that \vspace{-.1cm}
\beq \label{eq:delay}
4 \log m_k  \leq c_0 + \log  k
\eqs, \vspace{-.1cm}
\eeq
for all $k \geq 1$ and some $c_0\ge0$.
\end{Assumption}
Note that it follows that A\ref{ass:delay2} implies that $\{m_k\}_{k \geq 1}$
satisfies \vspace{-.1cm}
\beq
2 m_k \leq m_0 +
\big( (1/3) - \beta \big) k, \vspace{-.1cm}
\eeq
for proper selection of $m_0>0$ and $\beta\in (0,\frac{1}{3})$.
In the interest of space, the detailed proofs
for the below analysis can be found in our
online appendix (\texttt{https://arxiv.org/abs/1803.08198}).

Later, we will show that the distributed message passing
protocols in Section~\ref{sec:dist} satisfies A\ref{ass:delay} with high probability.
The following establishes
the linear convergence of SUCAG:
\begin{Theorem} \label{thm:main}
Assume that A\ref{ass:cts_h}, A\ref{ass:cts}, A\ref{ass:strcvx}, A\ref{ass:delay}
hold, and let\footnote{This is with absolutely no loss in generality, and is solely used in the analysis so as to simply notation in the derived stepsize selection rule.} $\mu \geq 1$.
Fix $\delta = 1 - \Delta \gamma \frac{{2}\mu L}{\mu+L}$ for some sufficiently small $\Delta \in (0,1)$,
and step size $\gamma$ that satisfies:
\beq \label{eq:gamma}
\begin{split}
& \gamma \leq \frac{ 2 }{ \mu + L}\\
& \gamma \leq \min_{j=1,...,4} \Big( \frac{3m_0 }{2} \frac{\mu L}{\mu + L} \Delta + \frac{1}{ C_j'} \Big)^{-1} \eqs,
\end{split}
\eeq
where the constants are given by:
\beq
C_j' \eqdef \frac{e^{1-c_0} \beta \Delta(1-\Delta) }{4 \tilde{q}_j \| \prm^0 - \prm^\star \|^{2 \tilde\eta_j-2} } \Big( \frac{2 \mu L}{\mu + L} \Big)^2,~j =1,...,4 \eqs,
\eeq
with
\beq \notag
\begin{split}
& \tilde{q}_1 \eqdef 2 \bar{L}_H L^2, \tilde{q}_2 \eqdef 32 \bar{L}_H^3, \tilde{q}_3 \eqdef  8 \bar{L}_H^2 L^4, \tilde{q}_4 \eqdef 2048 \bar{L}_H^6, \\
& \tilde\eta_1 \eqdef 3/2,~\tilde\eta_2 \eqdef 5/2,~\tilde\eta_3 \eqdef 2,~\tilde\eta_4 \eqdef 4 \eqs.
\end{split}
\eeq
Then, the  following properties hold:
\begin{enumerate}[leftmargin=.5cm]
\item The SUCAG
method converges linearly as $ \| \prm^k - \prm^\star \|^2 \le \delta^k \| \prm^0 - \prm^\star \|^2$
for all $k \geq 0$.
\item There exists an upper bound sequence $\{ \bar{V}(k) \}_{k \geq 0}$
such that $\| \prm^k - \prm^\star \|^2 \leq \bar{V}(k)$ for all $k \geq 0$ with
the rate:
\beq \label{eq:asympt}
\lim_{ k \rightarrow \infty} \frac{ \bar{V}(k+1) } {\bar{V}(k) } = 1 - 2 \gamma \frac{ \mu L }{ \mu +L} \eqs.
\eeq
\end{enumerate}
\end{Theorem}
First,
from Theorem~\ref{thm:main},
we consider the case when
$\| \prm^0 - \prm^\star \| \approx 0$.
In particular, taking
$\Delta = \| \prm^0 - \prm^\star\|^{\frac{1}{2}}$ shows that
the right hand side
of the second inequality in \eqref{eq:gamma}
is at the order of $\Omega( \| \prm^0 - \prm^\star \|^{-\frac{1}{2}} )$.
When $\| \prm^0 - \prm^\star \| \approx 0$, the SUCAG
method is allowed to take the step size at $\gamma = 2 / (\mu+L)$
in \eqref{eq:gamma}.
Substituting this into \eqref{eq:asympt} shows that
the asymptotic linear convergence rate of the SUCAG method
is $1 - 4 \mu L / (\mu + L)^2 = 1 -\Omega( 1/\kappa(F))$.
Consequently, the method finds an $\epsilon$-optimal solution
to \eqref{eq:opt} using just ${\cal O}( \kappa(F) \log(1/\epsilon) )$
steps.

Second, assumption A\ref{ass:delay} on the delays in the SUCAG method
allows for a sub-linear delay which
grows \emph{unboundedly} as fast as $m_k = {\cal O}(k^{1/4})$.
This is of relevance to the two
distributed implementations discussed in Section~\ref{sec:dist}.
In particular, in both cases
the agents' activations
can be captured by a finite state Markov chain, and the time delays can be
analyzed via the \emph{hitting time} of the chain:\vspace{-.1cm}
\begin{Lemma} \label{lem:mc}
Consider the sequence $( i_k )_{k \geq 0}$
that is governed by a finite, irreducible and aperiodic Markov chain, and suppose that the chain is
stationary\footnote{We note that the analysis applies even to non stationary Markov chains; we skip the details for length considerations.}.
It holds that
\hide{
\beq \label{eq:mc}
P( | k - \tau_i^{k-1} | > 1+x ) \leq {\rm exp} \Big( -\frac{x}{1 + e \bar{\tau}_i} + 1 \Big),~\forall~x \geq 0,
\eeq
}
\beq \label{eq:mc}
P( | k - \tau_i^{k-1} | > x ) \leq {\rm exp} \Big( -\frac{x}{1 + e \bar{\tau}_i} + 1 \Big),~\forall~x \geq 0,
\eeq
for all $i =1,...,N$, where $\bar{\tau}_i$ is the expected first visit time.
In particular, the  delay
$| k - \tau_i^{k-1}|$ can be bounded by ${\cal O}( \bar{\tau}_i + \bar{\tau}_i \log (Nk^2/\epsilon)$ with probability at least $1-\epsilon$
\end{Lemma}
In specific, for any connected undirected graph,
the Markov chain is stationary and $\bar{\tau}_i = N$
provided that the first activated agent is selected uniformly at random.
Consequently, even though $m_k \rightarrow \infty$ as $k \rightarrow \infty$, the time varying delay $m_k$ satisfies assumption A\ref{ass:delay}
w.h.p.

Importantly, combining Theorem~\ref{thm:main} and Lemma~\ref{lem:mc}
shows that when the initialization is sufficiently
close to the optimal solution, w.h.p.~the distributed implementation of SUCAG converges linearly at an asymptotic rate that is \emph{independent from the communication graph structure}.

\subsection{Proof of Theorem~\ref{thm:main}} \label{sec:pf}
The proof consists of two parts --- first we prove a descent lemma for the SUCAG
method to characterize its per-iteration progress; then we show that the
resulting nonlinear process converges linearly, with asymptotic rate as in \eqref{eq:asympt}.
Let us state the first result as follows:\vspace{-.1cm}
\begin{Lemma} \label{lem:descent}
Assume that A\ref{ass:cts_h}, A\ref{ass:cts}, A\ref{ass:strcvx}, A\ref{ass:delay} hold
and $\gamma \leq 2 / (\mu+L)$.
The following holds for all $k \geq 0$:
\beq \label{eq:lemma1}
\begin{split}
& \| \prm^{k+1} - \prm^\star \|^2 \leq \Big( 1 - 2 \gamma \frac{\mu L}{\mu + L}\Big) \| \prm^k - \prm^\star \|^2 \\
& +  \gamma^3 (m_k)^2  \max_{ (k-2m_k) \leq \ell \leq k } \Big( \tilde{q}_1 \| \prm^{\ell} - \prm^\star \|^3 +
\tilde{q}_2 \| \prm^{\ell} - \prm^\star \|^5 \Big) \\
 & + \gamma^6 (m_k)^4 \max_{ (k-2m_k) \leq \ell \leq k } \Big(
 \tilde{q}_3 \| \prm^\ell - \prm^\star \|^4 +
 \tilde{q}_4 \| \prm^\ell - \prm^\star \|^8 \Big)
 \\[-.2cm] \end{split}
\eeq
\end{Lemma}
An important fact observed is that the
distance to optimal solution at the $(k+1)-$th iteration
follows an inequality system  with high order terms
of the distance to optimal solution evaluated at \emph{delayed} time instances.

Let $R(k) \eqdef \| \prm^k - \prm^\star \|^2$,
the above motivates us to study the system:
\beq \label{eq:lem_ineq}
R(k+1) \leq p R(k) + \sum_{j=1}^J q_j  m_{1,j}^{(k)} \hspace{-.1cm} \max_{ (k - m_{2,j}^{(k)})_+ \leq \ell \leq k } \hspace{-.1cm}
R^{\eta_j}(\ell) ,
\eeq
where $p\in (0,1)$, $\eta_j > 1$ for all $j$ and $R(k)$ is non-negative.
Moreover, we have $m_{1,j}^{(k)}, m_{2,j}^{(k)} \geq 0$.
The following lemma establishes the linear convergence for $R(k)$
under a sufficient condition:
\begin{Lemma} \label{lem:nonlinear}
Fix $\delta \in (0,1)$  such that $\delta > p$.
Assume that
\beq \label{eq:delay_gen}
\log( m_{1,j}^{(k)} ) \leq c_{0} + c_{1} \log k,
m_{2,j}^{(k)} \leq m_0 + \Big( \frac{\eta-1}{\eta} - \beta \Big) k \eqs,
\eeq
for some $m_0, c_{0}, c_{1} \geq 1$, $0 < \beta < (\eta-1)/\eta$,
and $\eta \eqdef \min_{j=1,...,J} \eta_j > 1$.
Moreover, assume that
\beq \label{eq:lem_1_pre}
q_j \leq \frac{1}{R^{\eta_j-1}(0)} \frac{ \delta^{-{\xi}_j^\star(\delta)} (\delta - p) }{J} \eqs,~j=1,\hdots,J \eqs,
\eeq
with the non-positive number $\xi_j^\star(\delta)$ defined as:
\beq \label{eq:lem_2}
\xi_j^\star ( \delta) \eqdef \min_{ k \geq 0} \Big( \frac{ \log m_{1,j}^{(k)} }{\log \delta} + \eta_j \big( k - m_{2,j}^{(k)} \big)_+ - k \Big) \eqs.
\eeq
It holds for \eqref{eq:lem_ineq} that \emph{(i)} $R(k) \leq \delta^k R(0)$ for all $k \geq 0$,
and \emph{(ii)} there exists an upper bound sequence $\{ \bar{R}(k) \}_{k \geq 0}$ such that $\bar{R}(k) \geq R(k)$ for all $k \geq 0$ and its convergence rate is:
\beq
\lim_{ k \rightarrow \infty } \bar{R}(k+1) / \bar{R}(k) = p \eqs.
\eeq
\end{Lemma}
We remark that a crucial point to guaranteeing the linear convergence rate
is on the high order terms with power $\eta_j > 1$.
In fact, for $\eta_j = 1$ with some $j$, it can be shown
that $R(k)$ may diverge as $m_{1,j}^{(k)}, m_{2,j}^{(k)}$ is growing sublinearly in \eqref{eq:delay_gen}.

Finally,
we observe that \eqref{eq:lemma1} is a
special case of \eqref{eq:lem_ineq}
by substituting $R(k) = \| \prm^k - \prm^\star \|^2$ and the parameters:
\beq \nonumber
\begin{split}
& p = 1 - \frac{2 \gamma \mu L}{\mu+L},~\eta_1 = \frac{3}{2},~\eta_2 = \frac{5}{2},~\eta_3 = 2,~\eta_4 = 4 \eqs, \\
& q_1 = \gamma^3 \tilde{q}_1,~q_2 = \gamma^3 \tilde{q}_2,~q_3 = \gamma^6 \tilde{q}_3,~q_4 = \gamma^6 \tilde{q}_4 \eqs,\\
& m_{1,j}^{(k)} = (m_k)^{2 \lceil \frac{j}{2} \rceil},~m_{2,j}^{(k)} = 2m_k,~j=1,\hdots,4  \eqs,
\end{split}
\eeq
Under A\ref{ass:delay}, we observe that the above $m_{1,j}^{(k)}, m_{2,j}^{(k)}$
satisfy \eqref{eq:delay_gen} in
Lemma~\ref{lem:nonlinear} with the same $c_0$ .
Moreover, in the online appendix we show
that the step size $\gamma$ specified in \eqref{eq:gamma}
gives the set of $q_j$ that satisfy \eqref{eq:lem_1_pre}.
This concludes the proof.

\hide{
Bound that we use:
\nfc{c/p the analysis for uniform distribution}

\nfc{we also need a small lemma for MC large deviations: the two papers we found so far are in Dropbox: ToRead$\rightarrow$MC\_Chernoff}.}

\section{Numerical Experiments}
\begin{figure}[t]
\centering
\ifplainver
\resizebox{.6\linewidth}{!}{\input{./Tikz/cdc_test_case.tikz.tex}}
\else
\resizebox{.9\linewidth}{!}{\input{./Tikz/cdc_test_case.tikz.tex}}
\fi
\caption{Comparison of distributed optimization methods for solving the logistic regression problem \eqref{eq:logist} over a time varying graph
with $N=250$ agents, batch size $B=1$, and $d=51$ parameters. The optimality gap
against iteration number is obtained through averaging over $10$ trials (except for the deterministic CIAG method).}\vspace{-.3cm} \label{fig:sim}
\end{figure}

We evaluate the performance SUCAG on the learning task of training a linear classifier via logistic regression. In this problem, the $i-$th
component function is
defined as:
\beq \label{eq:logist}
f_i ( \prm ) \eqdef \sum_{b=1}^B \log( 1 + {\rm exp} ( -y_{i,b} \langle \prm, {\bm x}_{i,b} \rangle ) ) + \frac{B}{2N}\| \prm \|^2 \eqs,
\eeq
where $( {\bm x}_{i,b}, {\bm y}_{i,b} )_{b=1}^B$ represents
data held by agent $i$, and $B$ is the size of the corresponding data subset.
It can be verified that \eqref{eq:logist} and its corresponding sum function $F(\prm)$
satisfy A\ref{ass:cts_h}--A\ref{ass:strcvx}.
We generate a synthetic dataset in the experiment
where the ground truth classifier $\prm_0$ is selected as
$\prm_0 \sim {\cal U}[-1,1]^d$, then each feature vector ${\bm x}_i$
is generated independently following ${\bm x}_i \sim {\cal U}[-1,1]^{d}$
with the label $y_i$ computed as $y_i = {\rm sign} ( \langle {\bm x}_i, \prm_0 \rangle )$.
To simulate the distributed optimization methods, we generate $G$ as an Erdos-Renyi graph with $N$ agents and a connection probability of $2 \frac{\log N}{N}$ (where, if needed, we repeat the procedure until a connected graph is obtained).
The SUCAG method operates on the aforementioned graph using the protocol
in Section~\ref{sec:dist} (see also~Fig.\ref{fig:sucag}), where the probability
for selecting the next agent is uniform over all neighbors.
For benchmark purposes, we compare the SUCAG method to the similarly
modified version of distributed CIAG~\cite{CIAG} and SAG~\cite{SAG}.
Moreover, we compare the DIG-ing method \cite{nedic2017achieving},
an average consensus-based distributed optimization method that accounts for time varying communication topologies. For this method, the mixing matrix is designed in a similar fashion as the  pairwise gossip protocol \cite{boyd2006randomized}, while the  agents' activation sequence follows the same model  as in our proposed scheme.
The deterministic CIAG method \cite{CIAG} implemented on a fixed ring graph
is also compared for benchmarking purpose.

Fig.~\ref{fig:sim} shows
the numerical findings (averaged over 10 generated random walks) for a synthetic dataset example
with $d=51$ parameters, $N=250$ agents, where each
agent has a batch of $B = 1$ data tuple, \ie
there are $NB = 250$ data tuples in total.
For both SAG and DIG-ing methods, we have optimized their
step sizes (as $\gamma = 0.2/L$ and $\gamma = 0.02/L$, respectively)
used so that the methods converge in the trials
performed.
For SUCAG and distributed CIAG methods, two step size configurations are compared.
The numerical results show that the SUCAG method (as well as the distributed CIAG
method) outperform all other methods, and thus clearly demonstrating the benefits of
curvature-aided gradient tracking
in accelerating distributed optimization.

\section{Conclusions}
We have proposed an unbiased curvature-aided stochastic gradient method for large-scale optimization.
Our method is implementable in a distributed setting, both via distributed computations performed from agents activated by a single coordinator, as well as via a fully distributed random walk on the communication graph.
Under usual assumptions, our convergence analysis establishes linear convergence at a rate that solely depends on the condition number, but is \emph{independent from the network size} provided that the initialization is sufficiently close to optimality. Besides, we have established more general results under rather weak requirements, \ie   infinitely often activation, even when activation delays grow unbounded: our line of reasoning  may be applied to a wide class of stochastic optimization primitives. Our numerical experiments verified faster convergence compared to various methods for distributed and stochastic optimization.\vspace{.1cm}

\textbf{Acknowledgements}.
This work was supported by NSF under grants NSF CCF-BSF 1714672, CCF-1717207 and CCF-1717391. The authors would like to thank Prof. Maxim Raginsky for pointing out the reference~\cite{roch2015}.

redpajama_set_name: RedPajamaArXiv