text:
The City of Gold
As Sale Prices Deflate in Much of the Country, Apartment Sales in Manhattan Continue to Rise
Publication: New York Magazine
Agents: Samantha Kleier Forbes, Michele Kleier
Buy Low, Divorce High
Michele Kleier Discusses the Intersection of Divorce and Real Estate
FOR years, Michele Kleier, a real estate broker on the Upper East Side, knew why one of her most persistent clients was calling even before picking up the phone.
The client, a former high-ranking fashion executive and perpetual volunteer at her children's private schools, was checking the price she could get for her nine-room co-op in a prewar building. When the market reached a high, she told Ms. Kleier, she planned to divorce her husband, sell the apartment and live on her share of the profits.
Last year, Ms. Kleier delivered the long-awaited news: Manhattan luxury apartments were at a peak. The client went through with her plan. Now the woman calls from her new condo in California, raving about the weather and the distance from her ex-husband.
"She felt that she couldn't walk out on him until she had the money to move away and buy something on her own," Ms. Kleier said. "The real estate market allowed her to buy her freedom."
A little-noted side effect of the property boom of the past decade has been the real-estate-enabled divorce. Home values might have slid in some markets, but in the New York City region, where prices remain high, divorce professionals like therapists and lawyers, along with real estate brokers, say unhappily married couples are cashing in appreciated homes to underwrite a split.
"The equity that there is in real estate is one of the impetuses why there are so many divorces," said Nancy Chemtob, a Manhattan divorce lawyer, adding that the net worth of her clients has doubled in the past three years mainly thanks to real estate. The price of the average Manhattan apartment was $1.3 million as of June, up 7 percent from a year ago, according to the real estate brokers Brown Harris Stevens.
A spouse who has not worked, like Ms. Kleier's client, might decide that with a divorce settlement enriched by real estate, it is possible to maintain a comfortable standard of living. Or a breadwinning spouse might recognize that even after dividing community property, it will be possible to live well as a single person.
"No matter what the net worth of the client," Ms. Chemtob said, "the $3 million apartment is now the $7 million apartment, and the $7 million apartment is the $14 million apartment. Half of a lot is a lot."
That is how Sharon Sheinker thinks about the real estate equity she and her ex-husband accumulated over a 16-year marriage, which she said made the decision to legally separate easier.
The former couple made enough on their first apartment in New Jersey, and then on a second home on Long Island, to build a five-bedroom house in a gated community in Dix Hills, on Long Island, four years ago, for $1.1 million.
They recently signed a contract to sell the house for $1.4 million, less than the asking price of $1.579 million. But Ms. Sheinker calculates a $60,000 profit each year over the past four years. She will use her share to buy a smaller house in Dix Hills and continue to run a charity, A Gift From Alexa, in honor of her 6-year-old daughter, who is autistic.
"Money is freedom," Ms. Sheinker said. "I don't need the mansion. We made enough money to be able to get divorced and support two households."
Economists are familiar with this phenomenon. Even though divorce rates are declining over all, as far back as 1977 the economist Gary Becker showed that couples experiencing any unexpected, drastic rise in net worth are at risk of divorce. (The same holds true for a drastic decline in net worth.)
Extrapolating from survey data, Dr. Becker concluded in The Journal of Political Economy that "a greater deviation between actual and expected earnings increases the probability" of divorce.
Although couples who see their incomes rise steadily generally stay together, those who make more money than they ever expected are vulnerable to divorce. They realize that they are less financially dependent on each other and that they might have chosen different spouses if they had more choices at the time, said Dr. Becker, who teaches at the University of Chicago.
Dr. Becker, who won the Nobel Prize in 1992, also explored in his divorce study the economic argument for what many people today call trading up, or finding a trophy spouse.
Noting that 75 percent of men and more than 70 percent of women remarry within 15 years of a divorce, he found that divorced men with higher earnings have the greatest likelihood of remarrying. This implied, in his view, that men who have come into wealth have an incentive to divorce because they believe they could better their situation.
"They feel, given their status now, they can find other people of a type that appeals to them more than when they got married," he said in a telephone interview.
Kenneth Mueller, an East Village psychotherapist, says he has about a half-dozen clients who are real estate executives. Some, he said, have used windfall wealth from property to strengthen their marriages — like paying for counseling or adopting children. But others are emboldened to divorce and remarry. He said some men conclude that they can find a new spouse because their first wives were "not what I really wanted."
Of course, not all couples sitting on greatly appreciated homes are headed for divorce court. The likelihood of divorce depends on the strength of a marriage before the advent of unexpected wealth, according to Evelyn Lehrer, an economist who expanded on Dr. Becker's findings in 2003.
In her study "The Economics of Divorce," published in the book "Marriage and the Economy: Theory and Evidence From Industrialized Societies," Professor Lehrer concluded that couples who are likely to divorce after an unexpected change in assets often had weaker marriages to begin with. It is not the new wealth that causes the divorce; the money is just the catalyst.
Stephanie Coontz, the author of "Marriage, a History: From Obedience to Intimacy, or How Love Conquered Marriage," compared the current Gilded Age to an earlier one in the 1920s, when, she said, divorces spiked at a time of rapid wealth creation. In times when people accumulate wealth, she said, they often think they don't have to abide by society's conventional rules.
"When people get a lot of wealth in a hurry, it's more easy to act upon their impulses," she said. "You get used to sending back a steak because you don't like it. You send back a wife."
Real estate these days seems much on the minds of couples who are in counseling. Elyse Goldstein, an Upper East Side psychologist, said that half the couples she has seen in recent years have brought up real estate as a relationship issue, and 15 percent regarded it as a serious one.
"Real estate has become a language of emotional barter in terms of registering pains, hurts and resentments," Dr. Goldstein said. "If they can't have love, they can have real estate. There are a lot of fights about who is going to benefit from the windfall."
SUSAN KATZ, a fashion industry sales executive, said that real estate battles with her ex-husband over their home in Roslyn Heights, on Long Island, have dragged out over three years, involving three lawyers and $350,000 in legal fees.
She said that real estate was not the original cause of the divorce. But it has become the central issue as she and her ex-husband fight over what the house is worth, and how much she must pay to buy it from him.
Ms. Katz finds that some women who are divorcing can't cash in on the value of a home because they or their spouses borrowed against it. "Women think when they get divorced they can have half of the house," she said. "Most women don't realize how much this house is mortgaged."
And then there are cases in which couples decide a divorce settlement would ultimately be too costly because of the on-paper appreciation of their property.
One New York real estate executive, who has separated from his wife and would not speak on the record because he is unsure if he will divorce, said most of his peers in the industry who are unhappily wed seem to be staying put. They don't want to carve up the real estate portfolios they bought or built during the boom.
"I know plenty of people who are enormously wealthy and just don't want to cut it up," he said. "They find it hard to divide the real estate."
Publication: The New York Times


text:
Fresh moves making a return
This initiative lasted at least a year and a half hoping to provide relief to the food deserts of Chicago, Mayor Rahm Emanuel is hoping to bring this program back. When it shut down a few years ago it posted losses will it be successful in it's newest incarnation?
The program now is being run by Erika Allen, an Emanuel appointee to the Chicago Park District board and the director of Growing Power Chicago, an urban farming nonprofit that grows fruits and vegetables year round on 12 acres in the city. Allen will run Fresh Moves, with its first bus debuting July 18 and plans for a second bus to start up this fall.
In 2012, the U.S. Department of Agriculture provided a one-year grant of $45,000 to help Fresh Moves to begin operating a second bus. This year, Emanuel said Vilsack agreed to a one-year grant to serve as the seed money for the latest produce bus model. Allen said the grant is for $100,000, but she believes ultimately the bus program will largely fund itself through sales.
Asked what's different about the program this time around, Allen didn't hesitate: "We grow our own food."
Under Food Desert Action's previous business model, the foundation purchased produce and turned around and sold it on the buses, operating on thin price margins. Allen argued she has far less overhead than that previous setup since her nonprofit Growing Power, which started and still operates farms in Milwaukee, provides much of the food the bus will sell.
Allen said 90 percent of what is sold on the buses will be grown in Chicago or within 200 miles of the city, with the exception being more tropical fruits such as bananas and oranges. She said the bus also will sell salads and healthy drinks and snacks that will help generate revenue.
Labels: food desert, fresh moves, mayor, produce, Rahm Emanuel

redpajama_set_name: RedPajamaCommonCrawl


Sometimes everything in your life is destroyed and all you have left is ashes. Fortunately, as a Christian, God has a way of making something beautiful out of your ashes.
Whispers of smoke rose up as Cindy walked through the remains of her house. The fire, last night, completely destroyed her home. As she kicked a charred picture frame, she realized that these smoking embers are symbolic of what her life had become.
This past year, everything had fallen apart in Cindy's life. A misunderstanding with her boss caused her to lose her job. Her two-year relationship with her boyfriend had ended. Her car was destroyed when a drunk ran a stop sign. Now, her house has burned down.
The frustrations and pressures seem to be overwhelming. She asks herself, "How can I overcome this? Not only is there no light at the end of the tunnel, the tunnel seems to be collapsing all around me."
Have you ever felt this way? Have you ever felt that your life has turned into such a mess that all you have left are ashes? Most likely, as a Christian, you're probably familiar with the verses that talk about God being close to those in trouble. However, the old comfort of, "All things working together for the good" (Rom 8:28) may seem rather shallow. You ask yourself, "How can God create something good out of my life when all I have left are ashes?"
Many people don't realize if you mix ashes in oil and then put the mixture into a salt brine, you'll get a "scum" that floats to the surface. If you squeeze the scum together, you will end up with a bar of soap. What originally appeared to be a worthless waste product is actually a beneficial healing product.
Think about this concept for a moment. If you mix the ashes of your ruined dreams along with the anointing oil of God into the salt of your tears, you will end up with a useful substance. The "scum" you see floating in your life is not a worthless byproduct of a life-gone-bad. It is actually something that is very useful. God can take the remains of your destroyed dreams and turn them into something that can help cleanse those around you. God can use the lessons you have learned from your experiences to help heal dirty and wounded hearts.
Many of the problems we encounter are outside our control. We do, however, bring some problems on ourselves. Sometimes it is our sin that has caused our world to collapse. Does God rebuild our lives into something special even if it is our sins that has caused the destruction?
You may say, "Okay, I can see how God can take a life that is hurting and turn it into a shining example of God's love and power. I can't see, however, how God can salvage the testimony of a Christian who has allowed his life to be completely destroyed by sin. Once a Christian life has made a total mockery of the gospel and has alienated both Christians and non-Christians, what is left for him?"
That is a very good question; what is left for him? Unfortunately, we treat the repentance of a Christian and a non-Christian differently. If a person makes a total mess of his life and then becomes a Christian, we hold him up as a shining example of what God can do. If, however, a Christian falls into the same despicable sin and then repents, we usually try to hide him in a corner somewhere. He has become an embarrassment to us.
There is a double standard in the Christian community. While I think there are some practical considerations that require us to treat fallen Christians differently, I think a critical and judgmental spirit motivates most of our reactions to these Christians. The unfortunate reality for these fallen Christians is most ministries will not want to employ them. So, what is left for these people? Will the scars of their terrible sins prevent them from ever being used of God again? Sin always produces scars, and as a result, there is no way for things to completely return to the way they were before.
To illustrate, let's say you are a youth minister at your church and have fallen into deep sin. It has been discovered you have been having sexual relations with several of the young girls in your youth group. In fact, some of the girls are as young as eight years old. Many things will probably happen all at once. You will lose your job and the ministry of your church is set back many years. Your wife leaves you and your transgressions make national news.
After several years in prison, you finally turn your life around. Your repentance is genuine, and you have taken multiple steps to correct your deviant behavior. Your walk with God is closer than it has ever been. Your humility before God is profound.
After you get out of prison, you try to pick up the pieces only to find there aren't any pieces to pick up. Your family and all of your Christian friends want nothing to do with you. Your former pastor says it wouldn't be a good idea for you to be actively involved in your former church. In fact, he prefers you attend a different church. Most parents have said they don't want you to be near their children. Despite your burden to serve God, no one wants your help.
You understand their rejection, but this does little to ease your pain. You ask yourself, "What type of service is left for me?" You still have a burden to serve God, but no one will give you an opportunity. You think about the people who have been saved in prison and now have growing ministries. You think about the unfairness and you struggle to overcome the bitterness.
So, what do you do? I guess you could crawl into a hole somewhere and live out the rest of your life in shameful isolation. You could change your name and move to another state and start working in some distant church. Assuming they don't do a background check, it still wouldn't do you any good. God will not bless a ministry based on a lie. So, we go back to the original question, what do you do?
Rom. 5:20,21 says where sin abounds, grace abounds even more. Are these just hollow words that sound good in a sermon, or does God take special care of his fallen children? If this is true, the youth pastor should have received a great deal of extra grace.
God has provided, through nature, a beautiful illustration of His grace. Long ago in a forest there once stood a mighty tree. It was the tallest and most beautiful tree in the whole forest. It provided shade and shelter for a large variety of animals. One day the world came crashing in on the life of the tree (quite literally; an earthquake swallowed up the whole area). Now, the tree has no practical purpose and is basically worthless. To add insult to injury, conditions were just right to cause the tree to turn into coal. This mighty tree is now a dirty, messy substance, something that is undesirable to be around. To make matters worse, this tree was then put under incredible pressure. Amazingly, this destroyed tree, this filthy mess, has turned into a diamond.
The tree was designed for a specific purpose, but it could no longer function as a tree. Its original purpose and function had been completely destroyed. There was no turning back. Yet, this tree can still be useful. Even though it can no longer be used as it was intended, it now has a new function. Being a beautiful diamond, it can be used for something very special, such as a jewel on a king's crown.
Likewise, a Christian who has completely destroyed his testimony and ministry still has hope. He probably will never be able to be involved in the type of ministry he was involved in before. He can, however, be used in other types of service. It may not be a high profile position. It may not even appear glamorous to him or anyone else. Yet, in God's eyes, these remaining years may be even more valuable and productive than his previous years in his other ministry.
Isn't it miraculous to see how God can take a dark filthy mess and turn it into something beautiful and valuable? What do you think He can do with the mess you've made in your life? If you have fallen deep into sin, God can still make your life valuable and useful. You'll still have to live with the terrible scars of your sins, but these scars will be a living testimony of God's love and grace. Truly where sin abounds, grace does much more abound.


Skillweb, the mobile computing specialist company, has announced the availability of the latest in its family of wireless applications - Task Control Manager (TCM). TCM is aimed at mobile workers in retail delivery and fulfilment who need to receive tasks and be able to return status reports back to head office, specifically where delivery schedules need to be flexible to provide better customer service.
Housing Units, a home furnishings chain in Manchester, has commissioned a new retail system from leading Microsoft Dynamics Partner K3 Landsteinar, in order to grow the business. The order is worth approximately 450,000.
Windmill Hill Pro Golf Shop in Milton Keynes has become the first UK retail user of a new portable point of sale signage and promotions system that enables the shop to boost sales.
This multinational company has chosen Datalogic's hand held bar code reader, Gryphon, for many of its stores around the world.
Peter Jones, one of Londons leading department stores and part of the John Lewis Partnership, is the first UK retailer to trial zero power graphical electronic labels, in place of traditional printed labels.


text:
Biodiversity helps coral reefs thrive – and could be part of strategies to save them
Biodiversity, Conservation, Cutting Edge, Decoding Nature, Endangered
Coral reefs are home to so many species that they often are called "the rainforests of the seas." Today they face a daunting range of threats, including ocean warming and acidification, overfishing and pollution. Worldwide, more than one-third of all coral species are at risk of extinction.
By Cody Clements
I am one of many scientists who are studying corals to find ways of helping them survive and recover. As a recent report from the National Academies of Science, Engineering and Medicine shows, researchers are exploring many different strategies. Some, such as managed breeding to make corals more tolerant of stresses, are already being developed at small scales. Others, such as moving corals to colonize new areas, have not been tested yet.
My own work examines whether greater diversity of coral species on reefs can help corals survive and thrive. In a study published earlier this year, my colleague Mark Hay and I found evidence that the answer is yes. This finding could help to inform broader strategies for making coral reefs more resilient in altered oceans.
In nature, more is better
Are ecosystems healthier if they contain many species than if they harbor only a few? This is a central question in ecology. Generally, scientists have found that ecosystems with more diverse foundation species – those that define a system and are inseparable from it, such as trees in a forest – tend to be healthier and function better.
Until recently, no one had applied this test to coral reefs. But we do know that healthy coral reefs are diverse, structurally complex ecosystems dominated by corals. In contrast, reefs that have been damaged by stresses such as coral bleaching events tend to become simplified, less diverse landscapes, often dominated by seaweeds.
For our study we chose a reef area on the southwestern coast of Fiji's main island, Viti Levu, in the South Pacific. Many reefs along this coast have been heavily degraded by overfishing and other human-related activities, reducing coral cover and allowing seaweeds to dominate.
There are hundreds of coral species across the Pacific, but at smaller scales, we found just five species or fewer during preliminary surveys conducted on the degraded reef at our site. Since these conditions mirror what is happening to many reefs worldwide, we saw it as an ideal place to test whether coral diversity matters for the "new normal" that we expect to see on reefs of the future.
Underwater gardens
Our team created 48 concrete plots on the seafloor of the degraded reef, which served as the bases for experimental coral gardens. We created single-species gardens that each contained one of three coral species – Pocillopora damicornis, commonly known as cauliflower coral; Porites cylindrica, also known as yellow finger coral; and Acropora millepora, one of a number species known as staghorn corals. We also planted mixed gardens containing all three species.
We chose these corals because they are common to reefs across the Pacific and are representative of different coral families that have shown varying responses to a variety of harmful disturbances. In all, each garden contained 18 coral individuals, for a total of 864 corals.
A Porites cylindrica coral planted in our experimental gardens. Each coral was embedded within an upside-down soda bottle neck using epoxy, which allowed us to easily attach or remove them from the garden plots. Cody Clements, CC BY-ND
Experimental coral gardens on a degraded reef in Fiji. Gardens with a mix of coral species performed better than gardens containing only one species. Cody Clements, CC BY-ND
To assess each coral's performance as it grew, we needed to remove them from their plots periodically. So we cut off the tops of hundreds of soda bottles and planted an individual coral in the upside-down neck of each bottle with epoxy putty. We embedded the bottle caps into our concrete slabs so that we could easily unscrew each bottle neck to examine the coral it held, then screw it back into its base. Over 16 months we weighed the corals and tracked other measures of their well-being, including tissue death and colonisation of each garden by harmful seaweeds.
We consistently found that corals grown in mixed-species gardens performed better than those in single-species plots. Within four months, coral growth in the mixed-species gardens was even exceeding the best-performing single-species gardens. This suggests that different species may benefit each other in yet unknown ways, at least during early stages of a coral community's development.
Examples of single- and mixed-species coral gardens through time during our 16-month experiment. At four months, mixed-species gardens were outperforming single-species gardens in multiple ways – growing faster on average than even the best performing single-species gardens (Acropora millepora). By 16 months, growth was comparable between mixed-species and Acropora gardens, but aggregate performance of single-species gardens continued to lag behind their mix-species counterparts. Clements and Hay, 2019, CC BY-ND
Why is more better?
The next question is what drove the effects that we observed. We hope to investigate a number of leads in future experiments. For example, farmers commonly observe that planting a diverse mix of crops helps to reduce the spread of infectious diseases among individuals. Could the same be true for coral reefs?
Our initial findings offer both concern and hope for the future of coral reefs. If diversity is integral to coral well-being, then continued species loss could dramatically alter these ecosystems in ways that lead to further reef decline. How many parts can be removed from the "ecosystem engine" before it breaks down?
That said, many of the strategies in the National Academies reportinvolve using biodiversity – both at the genetic and species level – to enhance coral reef resilience. Examples include cross-breeding corals between populations; altering coral genes to give them new functions, such as higher heat tolerance; and moving stress-tolerant corals or coral genes to new locations.
Promising advances in technology, such as mapping coral reefs from the air, may also help researchers assess coral health and determine which species they contain. This baseline information may help better inform management and restoration efforts.
Corals are in trouble, but they aren't down for the count yet. Perhaps harnessing the power of their remaining biodiversity can help give them a fighting chance.
[ Expertise in your inbox. Sign up for The Conversation's newsletter and get a digest of academic takes on today's news, every day. ]
Cody Clements is a Postdoctoral Fellow at the Georgia Institute of Technology. This article is republished from The Conversation under a Creative Commons license. Read the original article. Feature Image – A healthy coral reef at Swains island, American Samoa. NOAA/NMFS/PIFSC/CRED Oceanography Team CC BY
biodiversity, climate change, conservation, coral, ecosystem, endangered, global warming, landscape, ocean, overfishing, resilient, science, sea, species, technology, underwater
PrevPreviousFor India's imperiled apes, thinking locally matters
NextWhat happens to an ecotourism town when the wildlife doesn't show?Next
Collateral Damage?
IndiaSpend
TheConversation
How We Used Cameras In The Sky To Track Invasive Plant Species
The Fire-Fighting Children Of The Khasi Hills And The Decline Of Traditional Farming In North-East India
As Threats Escalate, Snow Leopards Could Lose Endangered Status (And Why That Matters)
As Communities Rebuild After Hurricanes, Study Shows Wetlands Can Significantly Reduce Property Damage
India's Wildlife Crisis, And Why Hope Is So Important
Saving Kunds Of Vrindavan
Ladakh Responds To Tourism's Demands On Its Water
How Much Longer Will Goa Remain Otter Worthy?
The Uncertain Death Of King Coal
Keeping Global Warming To 1.5 Degrees: Really Hard, But Not Impossible
Kirtinasha To Vidrupa: The Lesser River Goddesses Of South Asia
Manas National Park Illustrates The Human Dimension Of Rhino Conservation
Does Forest Certification Really Work?
The Stilling: Global Wind Speeds Slowing Since 1960
Why Deadly, Worsening Flash Floods In Bihar's Border Villages Don't Get India's Attention
Agent Orange, Exposed: How U.S. Chemical Warfare In Vietnam Unleashed a Slow-Moving Disaster


So, you need to post some data securely to one of them new-fangled web apps.
# would like to avoid writing the data to a temp file.
# 2>/dev/null just suppresses output to STDERR.
# curl is very gabby with stderr for some reason.
# ...and you won't have to deal with the Open2 library.
# strings but I suppose most people would use <<EOS here.


On July 18th, I had the privilege of accompanying over a dozen Michigan Public School teachers on a journey to historical sites in Ohio, Michigan, and Ontario. The sites commemorate battles in the late 18th and early 19th centuries, in which Native Americans were heavily involved in territorial struggles between British and American forces. The trip was part of remarkable week-long itinerary for educators, put together by one of our Heritage Grantees to launch the project, "Anishnaabek – Odawa/Ottawa Journey Towards Understanding." Sponsored by the River Raisin National Battlefield Park Foundation, the project brought this group together in order to produce a pilot curriculum for middle-school students.
The curriculum will emphasize contemporary Michigan Native American tribes' perspectives on their ancestors' involvement in local and regional history. In the words of Eric Hemenway, Michigan Humanities board member and Director of Repatriation, Archives and Records for the Little Traverse Bay Bands of Odawa Indians: "It's about honesty." In other words, the project aims to include a more "honest" version of historical events in the curriculum—a more complete story, told from multiple angles, of what shaped the United States into what it is today. Because Native American history gets swept under the rug or distorted by outsiders, the project brings in tribal experts like Eric Hemenway to give voice to often overlooked perspectives.
Hemenway stresses the unique value in the opportunity for Native Americans to tell their own stories, bringing the nuances and feelings that dwell within the Native community into the history classroom. When combined with the expertise of academic historians, memories from Native communities create a powerful "hybrid" story that can change our perception of what history means to people—with special emphasis on the importance of place in Native American cultures. For contemporary Native American students, learning history in this way can help to counter the dehumanizing effects of having outsiders tell their story for centuries. This loss of ownership of historical memory parallels the more general sense of loss that tribes across North America have been shouldering for centuries—loss of territory, language, community integrity, and sources of livelihood that directly resulted from the westward expansion of the United States.
While there still remains an immense amount of work to be done for Native communities to rebuild in the various ways they desire, Hemenway is optimistic about some recent developments. For example, the 2015 renaming of Alaska's Mt. McKinley to Mt. Denali, reflecting the indigenous name for the mountain, has important implications for restoring a sense of sacred space. Such small victories are significant in that they begin to pull back the layers of conquest and appropriation that Native Americans have suffered, even if only symbolically. The same can be said of the public outcry against sports mascots caricaturing Native Americans, as well as Oregon's decision to outlaw the highly offensive and demeaning term, "squaw," which still figures in the names of landmarks, streets, and schools around the country.
Where historical monuments once provided a one-sided perspective on violent clashes between Europeans and Native Americans, some progress has been made towards commemorating losses on both sides. Hemenway noted that incidents that were once glossed as "massacres" might better be understood as revenge killings, in which Native Americans retaliated against settlers for the slaughter of their people and the destruction of their villages.
The new history curriculum, which includes class field trips to some of these sites, aims to give both Native and non-Native students ownership over their state's history. For Native students, including them more directly in the retelling of historical events is empowering. It can temper the alienation that many Native American students feel in school when the history they hear does not connect with the stories they learn at home.
For other students, establishing a well-rounded sense of their state's history can help dispel misconceptions about Native Americans—whom Hemenway believes are the most misrepresented and stereotyped population in the country. When he visits classrooms, as he frequently does, Hemenway says that students often want to know "Where is the Indian?" They expect to see moccasins and a feathered headdress. The opportunity to interact with a representative of a contemporary Native community, along with the new history curriculum developed by this Heritage Grant project, will lead to the type of steady, incremental changes that will ultimately improve communication and understanding across ethnic and cultural divides in the state of Michigan.


text:
\section{Introduction}
\subsection{Preliminaries}
In recent years, utilization of deep approaches to approximate the policies of reinforcement learning (RL) agents achieved numerous successes in wide range of applications such as playing Atari games \cite{dqn}, autonomous driving \cite{autonomous_driving}, path planning \cite{rrt_rl}, playing board games like chess, shogi \cite{chess}, go \cite{go} and even beating human players on StarCraft \cite{starcraft} \cite{openai_five}. Nevertheless, there are several issues regarding the function approximation in the deep reinforcement learning setting \cite{td3}. One of the problems resulting from the function approximation is the systematic estimation bias that prevents the learning agent from reaching the maximum performance and deep methods to be applied to the real-world problems \cite{sutton88}, \cite{off_policy}. The estimation bias on value estimates in value-based RL algorithms has been studied for discrete action spaces \cite{ddqn, insights_rl, precup_2001, max_min, averaged_dqn}. Furthermore, similar work in the continuous control domain for actor-critic methods is done for a subcategory of estimation bias, that is, overestimation bias \cite{td3}. This paper shows that in deterministic actor-critic methods that aim to overcome the accumulated overestimation bias and high variance, there exists an underestimation bias on the value estimates \cite{wd3}, \cite{tadd}. Our work addresses this issue from a probabilistic point of view and improves the current state-of-the-art performance on several continuous control RL tasks.

The estimation bias on the action value estimates in value-based deep reinforcement learning is usually studied in two categories: underestimation and overestimation. Overestimation bias, resulting from the maximization of noisy estimates in the standard Q-learning, induces an accumulated error through the learning stage \cite{thrun}. In a function approximation setting, such an estimation noise is unavoidable as deep neural networks represent the action-value functions \cite{td3}. This inaccuracy in the action values is further exaggerated due to the temporal difference learning \cite{sutton88}. On the other hand, underestimation bias is an outcome of Q-learning variants that aim to eliminate the accumulated overestimation bias and high variance on the value estimates \cite{max_min}, \cite{td3}. Even though the standard deep Q-learning modifications are shown to overcome the overestimation bias and high variance build-up, the existence of underestimated state-action values can still degrade the performance of an RL agent by assigning low values to ``good" state-action pairs, causing suboptimal and divergent behavior.

We begin by establishing that this underestimation phenomenon is present in delayed policy gradient that utilizes a pair of critics, namely, Clipped Double Q-learning \cite{td3}, in the continuous actor-critic settings. During training, the minimum of the estimates by two critics is used to construct the target Q-value in the temporal difference learning \cite{sutton88}. Unfortunately, taking the minimum of Q-values in learning the targets produces a consistent underestimation of Q-value estimates despite the decoupled actor and critics. To address this problem, we first show that variance of the reinforcement signals that the agent encounters during the learning phase increases the underestimation bias. Then, to overcome underestimation bias and improve the performance of the deterministic actor-critic methods, we introduce a deep Q-learning variant that combines the ideas behind Clipped Double Q-learning \cite{td3}, and Maxmin Q-learning \cite{max_min}, Triplet Critic Update. Our approach leverages the notion that along with two value estimators, the value estimate of an additional estimator can be used to construct an approximate upper and lower bound to the value estimate.

We build our modification on the state-of-the-art deterministic actor-critic algorithm, Twin Delayed Deep Deterministic Policy Gradient (TD3) \cite{td3}, and introduce the Triplet Critic Delayed Deep Deterministic Policy Gradient (TCD3) algorithm. Our deterministic actor-critic algorithm reduces the underestimation bias to a negligible margin and prevents the accumulated overestimation bias and high variance. We evaluate our algorithm on 12 continuous control tasks from OpenAI Gym \cite{gym}, where we show that Triplet Critic Update significantly improves the performance of TD3 \cite{td3}. For the reproducibility concerns, we run our experiments across a large set of seeds for the sake of a fair evaluation procedure.

\subsection{Prior Art and Comparisons}
In reinforcement learning, prior works on the function approximation error in terms of the estimation bias and high variance build-up have been studied by \cite{pendrith} and \cite{mannor}. Our work focuses on one of the outcomes of the function approximation error, namely, the underestimation bias on the action value estimates.

Several approaches reduce the effects of overestimation bias resulting from the function approximation and policy optimization in deep Q-learning. One of the successor works to the deep Q-learning \cite{dqn}, Double Q-learning \cite{ddqn}, proposed by \cite{insights_rl} and \cite{ddqn}, employs two independent action value estimators to obtain unbiased estimates of Q-values. \cite{max_min} modified the Double Q-learning \cite{ddqn} by utilizing multiple value estimators. This approach, Maxmin Q-learning \cite{max_min}, considers randomly selected value estimates of multiple critics, and a minimum of which is used in learning the Q-value target. Furthermore, methods that employ multi-step returns offer a trade-off between the variance build-up and accumulated estimation bias. Such methods are proven to be effective through distributed methods \cite{ackr}, \cite{impala}, approximate lower and upper bounds \cite{approximate_bounds}, and importance sampling \cite{precup_2001}, \cite{munos_safe_and_efficient_off_policy}. Another trade-off in these approaches is using a longer horizon rather than finding a direct solution to the accumulated error. \cite{petrik}, on the other hand, offers a solution to diminish the discount factor for reducing the contribution of each erroneous estimate.

The concern with the direct solution to the accumulated approximation error has been overcome by \cite{td3} for actor-critic settings. Their study shows the existence of overestimation bias and accumulated variance induced by the deep function approximation of Q-values. An extension of the Deep Deterministic Policy Gradient algorithm \cite{ddpg}, Twin Delayed Deep Deterministic Policy Gradient \cite{td3} which our method is built on, introduces a direct solution to the problems of the function approximation error through the utilization of two critics, delayed actor updates and target policy smoothing regularization. TD3 \cite{td3} is shown to produce state-of-the-art results by a large margin with a sufficient number of training iterations. Although the improvements introduced by \cite{td3} can tame the accumulated error, the usage of two critics induces an underestimation bias on Q-value estimates. To overcome the underestimation error, \cite{wd3} approaches the problem with a modification on the update rule for the critics in the TD3 \cite{td3}. In their approach, the target for the temporal difference learning \cite{sutton88} is computed through the weighted linear combination of the minimum and average of the two critics, Weighted Deep Deterministic Policy Gradient (WD3) \cite{wd3}. \cite{tadd} extends the WD3 \cite{wd3} by increasing the number of Q-networks to three. Both studies have shown that the weighted linear combinations of Q-networks can obtain more accurate action value estimates and higher evaluation returns.

\subsection{Contributions}
Our contributions are as follows:
\begin{enumerate}
    \item We show that if the agent receives reinforcement signals throughout the learning phase vary on a large scale, the estimation bias on the Q-value estimates severely increases.
    \item We demonstrate by empirical results that for the cases in which the overestimation and variance of the value estimates are eliminated, the underestimation bias can still degrade the performance of the RL agent in terms of the evaluation returns and convergence speed.
    \item We introduce a Double Q-learning \cite{ddqn} variant that upper and lower bounds the value estimates without any introduction of a hyper-parameter. In this way, we reduce underestimating to a negligible level and keep the overestimation and high variance build-up removed.
    \item Through an extensive set of experiments, we show that our method improves the convergence speed and performance of the state-of-the-art on 12 challenging OpenAI Gym \cite{gym} continuous control tasks.
\end{enumerate}

\section{Background}
Reinforcement learning paradigm considers an agent interacting with its environment to learn the optimal, reward-maximizing behavior. The standard reinforcement learning can be formalized by a Markov Decision Process (MDP) defined by the tuple $(\mathcal{S, A}, p_{M}, r, \gamma)$ that involves a state space $\mathcal{S}$, an action space $\mathcal{A}$, and transition dynamics $p_{M}(s' | s, a)$ for the MDP in interest denoted by $M$. At each discrete time step $t$, given an observed state $s \in \mathcal{S}$, the agent chooses an action $a \in \mathcal{A}$ with respect to its policy $\pi: \mathcal{S} \rightarrow \mathcal{A}$ which can be either stochastic or deterministic, and receives a reward $r$ from a reward function $\mathcal{R: S \times A \rightarrow} \mathbb{R}$, and then observes a next state $s'$. The cumulative reward which the agent tries to maximize is defined as the discounted sum of rewards $R_{t} = \sum_{i = t}^{T}\gamma^{i - t}r(s_{i}, a_{i})$ where discount factor $\gamma \in [0, 1)$ is scaling long-term rewards such that short-term rewards can be prioritized more.

Reinforcement learning aims to obtain the optimal policy $\pi^{*}_{\phi}$ parameterized by $\phi$ that maximizes the expected return $J(\phi) = \mathbb{E}_{s_{i} \sim p_{\pi}, a_{i} \sim \pi}[R_{0}]$. In continuous control domain, parametrized policies which are usually approximated by deep neural networks, can be updated by computing the gradient of the expected return $\nabla_{\phi}J(\phi)$. In an actor-critic setting, the policy $\pi$, also referred as the actor, can be updated via deterministic policy gradient (DPG) algorithm \cite{dpg}:
\begin{equation}
    \nabla_{\phi}J(\phi) = \mathbb{E}_{s \sim p_{\pi}}[\nabla_{a}Q^{\pi}(s, a) |_{a = \pi(s)} \nabla_{\phi}\pi_{\phi}(s)].
\end{equation}
The expected return when taking action $a$ after observing the state $s$ while following the policy $\pi$, $Q^{\pi}(s, a) = \mathbb{E}_{s_{i} \sim p_{\pi}, a_{i} \sim \pi}[R_{t} | s, a]$, is also called the critic or action-value function of the agent that values the quality of a state-action pair. The action-value function or the critic is used to evaluate a learning agent's current policy and improve the policy to obtain higher quality choices of actions, i.e., a better policy.

In Q-learning, when the transition probability of an environment is known, the state-value function, $Q^{\pi}$ can be estimated recursively through Bellman equation \cite{belmann} given the transition tuple $(s, a, r, s')$:
\begin{equation}
    Q^{\pi}(s, a) = r + \gamma\mathbb{E}_{s', a'}[Q^{\pi}(s', a')], \quad a' \sim \pi(s').
\end{equation}

For a large state space, the action value can be estimated by a function approximator $Q_{\theta}(s, a)$ parameterized by $\theta$. In the deep setting of Q-learning \cite{dqn}, the critic network is updated through the temporal difference learning by a secondary frozen network $Q'_{\theta}(s, a)$, also known as the target network. In this way, a fixed target $y$ can be achieved to update the critic network:
\begin{equation}
    y = r + \gamma Q_{\theta'}(s', a'), \quad a' \sim \pi_{\phi'}(s'),
\end{equation}
where the actions for the next state are chosen from a separate target actor-network $\pi_{\phi'}$ in the actor-critic setting for continuous action spaces. The target networks are either updated by a small proportion $\tau$ at each time step, $\theta' \leftarrow \tau\theta + (1 - \tau)\theta'$, called soft-update, or periodically to exactly match the current networks. Such an update rule is orthogonal to any deep reinforcement learning method that utilizes a separate target network. For instance, it can be applied to off-policy methods that sample mini-batches of transition tuples from the experience replay buffer \cite{exp_replay} for the update.

\section{The Underestimation in Deterministic Actor-Critic Methods}
\label{section:underestimation_problem}

In discrete action domains, overestimation due to the analytical maximization of action values is an evident and widely studied artifact \cite{ddqn, insights_rl, precup_2001, max_min, averaged_dqn}. For actor-critic settings, the existence and effects of the overestimation have been proven by \cite{td3} through the policy updates via gradient descent. In their approach, the minimum value of these critics is used to compute the target action value at each iteration along with delayed actor updates and smoothed target policy value. However, such utilization of the minimum operator to overcome the overestimation of the Q-values may introduce an underestimation bias \cite{td3}, \cite{wd3}, \cite{tadd}. We begin by proving through basic assumptions and statements that the underestimation phenomenon occurs in continuous control, actor-critic methods in environments with different reinforcement signals. Then, we introduce our modified target Q-value update rule to reduce the underestimation bias while staying in the ``safe zone" of function approximation error in the actor-critic setting.

In the TD3 algorithm \cite{td3}, the policy is updated with respect to the value estimates of one of two approximate critics. Without loss of generality, we assume that the policy is updated with respect to the first approximate critic, which is denoted by $Q_{\theta_{1}}(s, a)$ through the deterministic policy gradient. We show that in an environment with rewards that vary on a large scale, the target update rule for the Q-values induces an underestimation bias.

Let $\phi_{\mathrm{approx}}$ define the parameters from the actor update by the maximization of the first approximate critic $Q_{\theta_{1}}(s, a)$:
\begin{equation}
\text{\footnotesize$
    \phi_{\mathrm{approx}} = \phi + \frac{\alpha}{Z_{1}}\mathbb{E}_{s \sim p_{\pi}}[\nabla_{\phi}\pi_{\phi}(s)\nabla_{a}Q_{\theta_{1}}(s, a)|_{a = \pi_{\phi}(s)}],$\normalsize}
\end{equation}
\normalsize
where $Z_{1}$ is the gradient normalizing term such that $Z^{-1}||\mathbb{E}[\cdot]|| = 1$. As the actor is optimized with respect to $Q_{\theta_{1}}(s, a)$ and the gradient direction is a local maximizer, there exists $\xi$ sufficiently small such that if $\alpha < \xi$, then the \textit{approximate} value of $\pi_{\mathrm{approx}}$ by the first critic will be bounded below by the \textit{approximate} value of $\pi_{\mathrm{approx}}$ by the second critic:
\begin{equation}\label{eq:td3_like_underestimation}
    \mathbb{E}[Q_{\theta_{1}}(s, \pi_{\mathrm{approx}}(s))] \geq \mathbb{E}[Q_{\theta_{2}}(s, \pi_{\mathrm{approx}}(s))].
\end{equation}
Note that in the latter equation, the estimated action values by both critics are overestimated. Then, we can treat the function approximation error as a Gaussian random variable:
\begin{align}\label{eq:error_gaussian_def}
\begin{split}
    Q_{\theta_{1}}(s, a) - Q^{*}(s, a) &= G_{1} \sim \mathcal{N}(\epsilon_{1}, \sigma_{1}), \\
    Q_{\theta_{2}}(s, a) - Q^{*}(s, a) &= G_{2} \sim \mathcal{N}(\epsilon_{2}, \sigma_{2}).
\end{split}
\end{align}
By (\ref{eq:td3_like_underestimation}) and (\ref{eq:error_gaussian_def}), we have $\epsilon_{1} \geq\epsilon_{2} \geq 0$. Moreover, as the presence of the delayed actor updates, the mean function approximation errors by both critics are not very distant due to the decoupling the actor and first critic, i.e., $\epsilon_{1} - \epsilon_{2} \approx 0$. Then, by the first moments of the minimum of two correlated Gaussian random variables \cite{nadaraj}, the expected function approximation error for the Clipped Double Q-Learning algorithm \cite{td3} becomes:
\begin{equation}
    \mathbb{E}[\underset{i=1, 2}{\mathrm{min}}\{G_{i}\}] = \frac{\epsilon_{1} + \epsilon_{2}}{2} - \frac{\theta}{\sqrt{2\pi}},
\end{equation}
where $\theta \coloneqq \sqrt{\sigma_{1}^{2} + \sigma_{2}^{2} - 2\rho\sigma_{1}\sigma_{2}}$ and $\rho$ is the correlation coefficient between the error distributions, $G_{1}$ and $G_{2}$. The error Gaussian's are correlated since the critics are not entirely independent due to the use of the same experience replay buffer and opposite critics in learning the approximate targets \cite{td3}. If $\sigma_{1}, \sigma_{2} > \sqrt{\frac{\pi}{1 - \rho}}\epsilon_{1}$, then the action value estimate will be underestimated:
\begin{equation}
    \mathbb{E}[\underset{i=1, 2}{\mathrm{min}}\{Q_{\theta_{i}}(s, a)\} - Q^{*}(s, a)] < 0.
\end{equation}

It can be observed from the underestimation condition that for a highly correlated pair of critics, underestimation does not exist. However, because of the delayed policy updates, the correlation between the critics is regularized \cite{td3}. Thus, a weak or moderate correlation between the pair of critics is expected \cite{td3}, which increases the possibility of underestimation.

Although delayed policy updates and the minimization of the value estimates aim to reduce error growth, the variances of the value estimates are not eliminated since they are proportional to the variance of the future rewards and estimation errors \cite{td3}. In function approximation setting, the Bellman equation is never exactly satisfied, yielding erroneous value estimates as a function of the true TD-error \cite{off_policy}, as expressed by (\ref{eq:error_gaussian_def}). Then, it can be shown that the variance of the value estimates overgrow as the agent interacts with the environment and observes varying reward signals \cite{sutton2018reinforcement}. We can express the approximate Q-values in terms of the expected value of the discounted cumulative rewards, as shown in \cite{td3}:
\begin{align}
\begin{split}
\text{\small$
    Q_{\theta_{i}}(s, a) = \mathbb{E}_{s_{i} \sim p_{\pi}, a_{i} \sim \pi}[\sum_{i = t}^{T}\gamma^{i - t}r_{i}] + \epsilon_{i}\sum_{i = t}^{T}\gamma^{i - t}.$\normalsize}
\end{split}
\end{align}
Suppose the expected value of the estimation error is constant for both critics. In that case, varying reinforcement signals increase the variance of the value estimates, which results in an underestimation error. The agent extensively explores the environment, a mandatory requirement for continuous action domain \cite{off_policy}, the variance of the reward signals that the agent encounters start growing, and underestimation bias will become inevitable. Furthermore, the minimum operator eliminates the accumulating error due to the temporal difference learning, and thus, underestimation is far more preferable to overestimation bias in actor-critic setting \cite{td3}. Nevertheless, underestimating a value estimate may discourage the agent from choosing good state-action pairs for an extended period and reinforce the agent to value suboptimal state-action pairs more.
\begin{figure*}[htbp]
	\centering

	\subfigure[Ant-v2]{
		\includegraphics[width=2in, keepaspectratio]{Q-Estimation Results/Q-Est_TD3_vs_TCD3_Ant-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{-0.06in}%
	\subfigure[BipedalWalker-v3]{
		\includegraphics[width=2in, keepaspectratio]{Q-Estimation Results/Q-Est_TD3_vs_TCD3_BipedalWalker-v3.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{-0.06in}%
	\subfigure[Humanoid-v2]{
		\includegraphics[width=2in, keepaspectratio]{Q-Estimation Results/Q-Est_TD3_vs_TCD3_Humanoid-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{-0.06in}%
	\subfigure[HumanoidStandup-v2]{
		\includegraphics[width=2in, keepaspectratio]{Q-Estimation Results/Q-Est_TD3_vs_TCD3_HumanoidStandup-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{0.04in}%
	\subfigure[LunarLanderContinuous-v2]{
		\includegraphics[width=2in, keepaspectratio]{Q-Estimation Results/Q-Est_TD3_vs_TCD3_LunarLanderContinuous-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{0.04in}%
	\subfigure[Swimmer-v2]{
		\includegraphics[width=2in, keepaspectratio]{Q-Estimation Results/Q-Est_TD3_vs_TCD3_Swimmer-v2.jpeg}
	}

	\caption{Measuring estimation bias of fine-tuned TD3 versus TCD3 while learning on MuJoCo and Box2D environments over 1 million time steps. Estimated and true Q-values are computed through Monte Carlo simulation for 1000 samples.}
	\label{td3_tcd3_q_estimation}
\end{figure*}

\subsection{Does the theoretical underestimation due to the minimum operator occur in practice?}
This question can be answered by observing value estimates produced by the target Q-value update with the minimum of two critics over a training duration along with the actual Q-values while the agent is learning on a set of OpenAI Gym \cite{gym} continuous control tasks. The true Q-values are estimated through the average discounted sum of rewards over randomly initialized 1000 episodes following the current policy. Estimated Q-values are also computed using the average Q-value estimates by the critics on the 1000 sampled episodes.

In Fig. \ref{td3_tcd3_q_estimation}, there exists an apparent underestimation bias that occurs on the learning procedure. The underestimation bias keeps growing or follows a pattern parallel with the actual values depending on the environment. These empirical findings verify our claims; that is, the approximate critics start by overestimating the actual Q-values. However, after a while in which the agent starts an extensive exploration procedure, the variances of the approximate critics increase, and an unavoidable underestimation arises. Moreover, for some environments such as Ant, as the actual state-action values increase due to the varying rewards, the variances of the approximate critics increase, and the underestimation on the value estimates keeps growing even with the delayed target and actor updates. Whereas, when the true Q-values reach a steady-state, the underestimation settles to a certain level which can also be observed from Humanoid, Swimmer, and BipedalWalker environments. Even though in the set of OpenAI Gym \cite{gym} tasks, the continuous state and multi-dimensional action spaces are contributing factors to the growth in the variance of the action-values, the scale of these tasks is still tiny compared to the real-world settings \cite{off_policy}. Therefore, underestimating value estimates would be a more detrimental and inevitable problem for tasks with larger-scaled state and action spaces.

To remedy the observed underestimation bias, we next introduce our novel modification on the target Q-value update, Triplet Critic Update, which significantly reduces the underestimation bias by the minimum of two approximate critics.

\section{Triplet Critic Update}
Several approaches have been proposed to overcome the underestimation bias introduced by the minimum of two critics. These approaches differ in terms of the treatments of the function approximation error as random variables. Although these proposals can overcome the underestimation bias introduced by the target Q-value update, they introduce additional hyper-parameters to be tuned. This section presents our parameter-free, novel modification on the target update rule for deterministic actor-critic methods for continuous control, which is also a variant of Double Q-learning \cite{ddqn} but with three critics.

In the current approach to overcome the overestimated action values, we highlight that the bias introduced by the approximate critics is a function of the expected function approximation error by the critics and variance of the value estimates. As the expected function approximation error is assumed to be constant for both critics, depending on the variance of the value estimates, which is proportional to the encountered reinforcement signals, the function approximation error can either be an underestimation or overestimation. In practice, however, we observe that the variance of the value estimates is much greater than the expected function approximation error as the continuous control tasks require a comprehensive exploration procedure. Hence, an underestimation error occurs caused by the objective computation for the action value functions that employ the minimum operator as in TD3 \cite{td3}. As a result, the critic does not value good state-action pairs as much as they should be.
To address this problem that has been approached from different perspectives, we propose to simply upper and lower-bound the biased value estimates of three critics without introducing any additional hyper-parameter. This results in taking the minimum of the maximum of two critics and a single distinct critic, which gives the modified target update rule, Triplet Critic Update:
\begin{equation}
\text{\small$
    y = r + \gamma \mathrm{min}\left(\underset{i = 1, 2}{\mathrm{max}}(Q_{\theta'_{i}}(s', \pi_{\phi'}(s')), Q_{\theta'_{3}}(s', \pi_{\phi'}(s')\right).$\normalsize}
\end{equation}
As the actor is optimized with respect to the first critic, the second and third critics can be represented by the same probability distribution, i.e., $G_{3} \sim \mathcal{N}(\epsilon_{2}, \sigma_{2})$. Under the same assumptions presented in the previous section, the expected function approximation error for the Triplet Critic Update can be computed by the extensions of \cite{nadaraj} and \cite{expect_of_moments_of_max}:
\begin{align}
\begin{split}
    \text{\small$\mathbb{E}[\mathrm{min}(\mathrm{max}(G_{1}, G_{2}), G_{3})]$} &= \text{\small$(\epsilon_{1} + 3\epsilon_{2}) / 4 - \theta / 2\sqrt{2\pi}$}, \\ &= \text{\small $\frac{(\mathbb{E}[\underset{i=1, 2}{\mathrm{min}}\{G_{i}\}] + \epsilon_{2})}{2}$\normalsize}.
\end{split}
\end{align}
This expected error value is the average of the biases introduced by the minimum of two target action values as in TD3 \cite{td3}, and an approximate critic as in DDPG \cite{ddpg}. If the overestimation is relatively larger than the underestimation, there will be a slight overestimation by this update rule, and vice versa for a relatively larger underestimation. Regardless, such slight estimation errors can be tolerated by the agent compared to TD3 \cite{td3} and DDPG \cite{ddpg}. However, in practice, the variance of the value estimates is usually large, and hence, the Triplet Critic Update will underestimate the value estimates. Nonetheless, the bias will be slightly larger than half of the bias in TD3 \cite{td3} which will greatly reduce the underestimation and obtain more accurate value estimates without introducing additional hyper-parameter to be tuned. In addition, as the underestimation is preferable to the overestimation due to non-accumulating error characteristics, \cite{td3}, the Triplet Critic Update remains in the ``safe zone" of the function approximation in general, by keeping the overestimation and accumulated variance eliminated. Finally, it is observable in our simulations that for the extreme cases in which the variance of the value estimates is very large or very small, the Triplet Critic Update offers more accurate estimates than TD3 \cite{td3} for both ends.

\begin{algorithm}[htbp]
    \caption{TripletCriticUpdate}
    \begin{algorithmic}
    \small
        \STATE \textbf{Input} $Q_{\theta_{1}'}, Q_{\theta_{2}'}, Q_{\theta_{3}'}$, $s'$, $\tilde{a}$
        \STATE $y \leftarrow r + \gamma \mathrm{min}(\mathrm{max}(Q_{\theta'_{1}}(s', \tilde{a}), Q_{\theta'_{2}}(s', \tilde{a})), Q_{\theta'_{3}}(s', \tilde{a}))$
        \RETURN $y$
    \end{algorithmic}
    \normalsize
    \label{alg:triplet critic update}
\end{algorithm}
\begin{algorithm}[hb]
    \caption{TCD3}
    \begin{algorithmic}
    \small
        \STATE Initialize critic networks $Q_{\theta_{1}}, Q_{\theta_{2}}, Q_{\theta_{3}}$, and actor network $\pi_{\phi}$ with randomly initialized parameters $\theta_{1}, \theta_{2}, \theta_{3}, \phi$
        \STATE Initialize target networks $\theta_{i}' \leftarrow \theta_{i}, \phi' \leftarrow \phi$
        \STATE Initialize replay buffer $\mathcal{B}$
        \FOR{$t = 1$ \textbf{to} $T$}
            \STATE Select action with exploration noise $a \sim \pi_{\phi}(s) + \eta$, $\eta \sim \mathcal{N}(0, \sigma)$ and observe reward $r$ and new state $s'$
            \STATE Store transition tuple $(s, a, r, s')$ in $\mathcal{B}$
            \STATE Sample mini-batch of $N$ transitions $(s, a, r, s')$ from $\mathcal{B}$
            \STATE $\tilde{a} \leftarrow \pi_{\phi'}(s') + \eta; \eta \sim \mathrm{clip}(\mathcal{N}(0, \tilde{\sigma}), -c, c)$
            \STATE $y \leftarrow $ \textbf{TripletCriticUpdate}$(Q_{\theta'_{1}}, Q_{\theta'_{2}}, Q_{\theta'_{3}}, s', \tilde{a})$
            \STATE Update critics $\theta_{i} \leftarrow \mathrm{argmin}_{\theta_{i}}N^{-1}\sum(y - Q_{\theta_{i}}(s, a))^{2}$
            \IF{$t$ mod $d$}
                \STATE Update $\phi$ by the deterministic policy gradient:
                \STATE $\nabla_{\phi}J(\phi) = N^{-1}\sum\nabla_{a}Q_{\theta_{1}}(s, a)\vert_{a = \pi_{\phi(s)}}\nabla_{\phi}\pi_{\phi}(s)$

                \STATE Update target networks:
                \STATE $\theta_{i}' \leftarrow \tau\theta_{i} + (1 - \tau)\theta_{i}'$
                \STATE $\phi' \leftarrow \tau\phi + (1 - \tau)\phi'$

            \ENDIF
        \ENDFOR
    \normalsize
    \end{algorithmic}
    \label{alg:tcd3}
\end{algorithm}

In the implementation, for the computational efficiency, the actor again should be optimized with respect to the first critic, and the utilization of more than three critics should be avoided. Therefore, the order of the critics in the nested Triplet Critic Update must remain constant. Otherwise, instability may occur if the critic with respect to which the actor is optimized and the order of critics in the update rule is altered throughout the learning phase. However, the utilization of three critics introduces additional computational complexity. If the clipped Double Q-Learning algorithm \cite{td3} is assumed to have a runtime of $\mathcal{O}(n)$, then our approach would run on $2\mathcal{O}(n)$ which increases linearly. Thus, the runtime can be reduced to $2\mathcal{O}(n) \approx \mathcal{O}(n)$ and neglected if the number of training time steps is not very large. Consequently, our modification comprehensively considers computational complexity and estimation error accuracy trade-off by introducing only a single additional critic without any hyper-parameter that requires further tuning.

We now introduce our approach built on the TD3 algorithm \cite{td3} changing the target Q-value update. Our algorithm called Triplet Critic Deep Deterministic Policy Gradient (TCD3) is summarized in Algorithm \ref{alg:tcd3}. In the next section, we present the experimental results for our algorithm in terms of both the Q-value estimation comparisons with TD3 \cite{td3}, and evaluations on several OpenAI Gym \cite{gym} continuous control environments.

\section{Experiments}
To evaluate our estimation error correction approach, we first demonstrate the estimated and actual Q-value curves for TCD3, and state-of-the-art off-policy continuous control algorithm TD3 on both MuJoCo \cite{mujoco}, and Box2D \cite{box2d} continuous control tasks interfaced by OpenAI Gym \cite{gym}. Subsequently, we evaluate and compare the learning RL agents under TCD3 and TD3 algorithms on the extended set of OpenAI Gym \cite{gym} control tasks. To provide reproducibility and allow a fair comparison, we directly use the same set of tasks from \cite{gym} with no modifications to the environment dynamics.

\begin{figure*}[t]
	\centering
    \textcolor{custom_blue}{$\blacksquare$} Fine-tuned TD3 \quad \quad \textcolor{custom_orange}{$\blacksquare$} TCD3

	\subfigure[Ant-v2]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/Ant-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{-0.04in}%
	\subfigure[BipedalWalker-v3]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/BipedalWalker-v3.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{-0.04in}%
	\subfigure[HalfCheetah-v2]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/HalfCheetah-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{-0.04in}%
	\subfigure[Hopper-v2]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/Hopper-v2.jpeg}
	}
	\hspace{-0.1in}%
	\vspace{-0.04in}%
	\subfigure[Humanoid-v2]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/Humanoid-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{-0.03in}%
	\subfigure[HumanoidStandup-v2]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/HumanoidStandup-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{-0.03in}%
	\subfigure[InvertedDoublePendulum-v2]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/InvertedDoublePendulum-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{-0.03in}%
	\subfigure[InvertedPendulum-v2]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/InvertedPendulum-v2.jpeg}
	}
	\hspace{-0.1in}%
	\vspace{-0.03in}%
	\subfigure[LunarLanderContinuous-v2]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/LunarLanderContinuous-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{-0.04in}%
	\subfigure[Reacher-v2]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/Reacher-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{0.04in}%
	\subfigure[Swimmer-v2]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/Swimmer-v2.jpeg}
	}
	\hspace{-0.3in}%
	\vspace{0.04in}%
	\subfigure[Walker2d-v2]{
		\includegraphics[width=1.7in, keepaspectratio]{Evaluation Results/Walker2d-v2.jpeg}
	}
	\caption{Learning curves for the set of OpenAI Gym continuous control tasks. The shaded region represents half a standard deviation of the average evaluation over ten trials. Curves are smoothed uniformly with a sliding window of 20 for visual clarity.}
	\label{evaluation_results}
\end{figure*}

\subsection{Implementation Details}
For the implementation of TD3 \cite{td3}, we utilize the open-source implementation from the author's GitHub (\url{https://github.com/sfujim/TD3}). The TD3 \cite{td3} implementation that we use is the most recent and fine-tuned version of the algorithm updated by the author as of April 2021. Our modification is built on top of the TD3 \cite{td3} implementation such that the number of critics and target Q-value computation are replaced.

\subsection{Experimental Setup}
We perform true and estimated Q-value comparisons for our approach and TD3 \cite{td3} over 6 OpenAI Gym \cite{gym} continuous control tasks are presented in Fig. \ref{td3_tcd3_q_estimation}. Each task is run for 1 million time steps, and curves are derived through the same procedure as explained in section \ref{section:underestimation_problem}.

Our evaluations on each task are performed by rerunning both algorithms over 1 million time steps and evaluating the performance of the agent every 5000 time steps without exploration noise. Each evaluation report is the average of 10 episode rewards on the distinct evaluation environment. The results are reported over 10 random seeds of the Gym \cite{gym} simulator, network initialization, and code dependencies.

\subsection{Discussion}
TCD3 obtains more accurate Q-value estimations than TD3 \cite{td3} on all environments. We observe two cases from our empirical results. First, our approach accurately estimates the Q-values by a negligible margin. Second, albeit the TCD3 greatly reduces the estimation bias, there still exists an underestimation error which is slightly bigger than half of the underestimation in TD3 \cite{td3}. These findings support our claim that an increasing variance of the reward signals encountered by the agent increases the underestimation, for example, in the Ant environment. Nonetheless, the expected value of the bias can be further reduced, as shown in this paper.

The evaluation results on the same set of tasks are depicted by Fig. \ref{evaluation_results}. Our algorithm either outperforms or matches the TD3 algorithm \cite{td3} in terms of both learning speed and final performance. We observe that the estimation error prevents the agent from reaching higher possible reward potentials and stable returns. On top of the improvement that the minimum of two approximate critics can eliminate the overestimation bias, our approach obtains higher and smoother evaluation returns by reducing underestimation to a negligible margin and keeping the overestimation eliminated.

These simulation results also demonstrate that the underestimation induced by the minimum of two critics can cause ``good" state-action values to be assigned low values, resulting in slower convergence and suboptimal action choices to be selected more frequently \cite{off_policy}. For example, from Fig. \ref{td3_tcd3_q_estimation} and \ref{evaluation_results}, we observe that for BipedalWalker and Humanoid environments, elimination of estimation bias yields faster convergence to the optimal evaluation returns. Overall, by significantly reducing this underestimation phenomenon for deterministic continuous control actor-critic methods, we show in this paper that agents can attain higher evaluation returns in fewer time steps with no estimation bias.

\section{Conclusion}
Accumulated overestimation bias induced by the function approximation in deep reinforcement learning has been identified as a substantial issue. On the contrary, in deterministic actor-critic settings, techniques to overcome function approximation error build-up may lead to underestimation bias which has been a problematic drawback. In this work, we first show that encountering different reward signals increases the underestimation bias. Then, we develop a novel variant of deep Q-learning that significantly reduces the underestimation bias to a negligible level. Taken our claim and empirical results together, this improvement defines our efficient, parameter-free update rule, Triplet Critic Update, which dramatically improves both the learning speed and performance of TD3 in several challenging continuous control tasks. Our algorithm, Triplet Critic Delayed Deep Deterministic Policy Gradient (TCD3), outperforms state-of-the-art by obtaining more accurate Q-value estimates. Our modification is orthogonal and can easily be adapted to any deterministic actor-critic method that employs temporal difference learning.

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\bibliographystyle{IEEEtran}

redpajama_set_name: RedPajamaArXiv


text:
module.exports = function(grunt) {
  grunt.loadNpmTasks('grunt-mocha-test');
  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    mochaTest: {
      test: {
        options: {
          reporter: 'spec'
        },
        src: ['test/*.js']
      }
    }
  });

  grunt.registerTask('test', ['mochaTest:test']);
};

redpajama_set_name: RedPajamaGithub


I pulled out some of my vintage catalogues today. What fun they are to look through. These images are from a Sears Spring & Summer 1938 mail order catalogue. The catalogue itself is about as thick as a modern phonebook and has everything you could ever need for your home or wardrobe. The beach outfits and swimsuits are all on a grand two pages. Go figure?!? There are about a dozen pages just focused on girdles and corset's. I also spent some time gazing at all the hats. But less I digress, onto the swimsuits!
Perfect 1938 model of real high grade. Mast of fast-colored, pre-shrunk cotton twill in a gloriously bright Tropical print. Smart promenade coat flares gracefully from fitted waist. Swim suit has wool jersey lining in matching color. Coat and swimsuit $4.95, the lined swimsuit alone $2.94.
b)Shimmering satin Lastex stretches crosswise for figure control. Woven or Rayon, cotton and Lastex. Skirtless style with cotton lined front, deep sun back. $2.98.
d) All wool. Our famous classic 'skirted' suit. Of all worsted wool. Shaped lined bust, moderate sun back. $1.54.
Really, aren't the beach shoes just adorable?
g)the top one is a zip-fastened detachable shirt of Zephyr wool. Heavy 2 ply wool worsted trunks $2.98.
j)Athletic cut white Zephyr wool shirt, zip fastened to trunk. all worsted wool $1.98.


Four "Honor Men Awards" are presented at each Hey Day celebration: the Spoon, Bowl, Cane and Spade. The recipients of each award are elected by the senior class based on their achievements and contributions to class life, which distinguished them as being the outstanding men in their class. In an enhancement of the Hey Day tradition, the winners of twenty-five years standing generally present the Senior Honor Awards.
The oldest of the awards is the Spoon, which originated in the mid-19th century. In the early 1860s, when the University was located on Ninth Street, the school year was divided into trimesters, with first trimester marks announced right before Christmas recess. Although there were few "first honor" or "second honor" men, there were usually many holding "third honors". In 1861, several joyous sophomores, imbued with the holiday spirit, presented the lowest freshman in the "third honor" group with a large wooden spoon. The spoon was engraved with a sarcastic inscription in Latin. Four years later, at the first Class Day, a spoon was presented to the most popular member of the senior class, who thereby became the "Spoon Man". For the first fifty years, the spoons were carved by master carver and furniture-maker Daniel Pabst.
The Bowl is the second oldest award, dating to 1882. Initially, a replica of the bowl was presented to one of the popular sophomores at the end of the Bowl Fight. By 1884, however, the presentation of the Bowl on Class Day was fixed as the second Senior Award.
The Cane Award was introduced to commemorate the Cane Fights, which began in 1890 and only lasted five years. The first presentation of the Cane as the third Senior Award took place in 1891. The origin of the Cane Fights is interesting. In the 1880s, when the cane was the hallmark of every young gentleman, the sophomores were infuriated that the lowly freshmen were allowed to carry canes.
They decided to take the law into their own hands by seizing the freshmen's canes and breaking them in half. After several years, the classes decided to formalize the custom by holding a cane fight. In these fights, the cane man dropped a cane in the middle of a field and both classes rushed toward it. After a specified time, the hands on the cane were counted and the winning class declared. At the Hey Day celebration, the Cane Award was presented to the third most popular senior.
The final Senior Honor Award is the Spade. The first presentation of the Spade as the fourth Senior Award took place in 1894. This ceremony formalized a custom which had grown since the inception of Ivy Day in 1873. Each year the student who used a spade to plant the class ivy was allowed to keep the tool as an expression of his classmates' esteem.
In addition to the Senior Honor Awards, many other awards and citations were given on Hey Day. These awards were presented to individuals and fraternities which had achieved distinction in leadership, ability, integrity and perseverance. Following the presentation of awards was the announcement of elections to membership in senior honor societies.


text:
How To Explore Cuba Through Ernest Hemingway's Most Famous Novel
March 19, 2017 August 12, 2017 by Simons Chase
Seeing the world through the lens of a great artist is a rare experience, but today there is an immersive way to combine art and travel if you are willing to explore a little.
Visit Cuba and gain the sensation of the Cuba that Nobel prize winning author Ernest Hemingway experienced as a backdrop to some of his more important works.
In Havana, a rich array of earthy colors is detonated as the bright Caribbean sun interacts with the city's majestic but fading structures. The city is one of the last remaining urban environments unspoiled by the aesthetic pollution of modern city advertising. There you can time travel back to Hemingway's era. Cuba's relative isolation from the West and in recent decades its reliance on preserved American cars dating from the 1950s means the city looks much like it did in 1959 just before Hemingway left the island nation.
Image by Cuba Journal contributor Mimi Fuenzalida
Havana's oily night is mystical from almost every block you visit. The lack of light pollution from excessive urban lighting creates a brilliant night sky – no different from the nights when Hemingway made the rounds to his favorite Havana drinking holes.
An Oily Night in Havana. By Simons Chase/Cuba Journal.
In fact, in Hemingway's short novel, The Old Man and the Sea, he claimed fraternity with the night sky. He says, through the main character, Santiago, "I am as clear as the stars that are my brothers. Still I must sleep. They sleep and the moon and the sun sleep and even the ocean sleeps sometimes on certain days when there is no current and a flat calm."
Except for the permanent statute of Hemingway at the bar, the Floridita is almost exactly the same as it was when Hemingway made it famous. Go early to beat the crowds.
El Floridita Bar-Restaurant
Hemingway's home, the Finca Vigia, sits on a bucolic country side hilltop overlooking Havana 15 miles away. The Cuban government turned it into a museum in the 1960s and today visitors can see the home and grounds exactly as Hemingway left it – down to the bottles of rum on the table next to his favorite chair. Here is an interview the Cuba Journal conducted with Hemingway's late chef, Fico.
Finca Vigía, Museo Hemingway
The fishing village of Cojimar, not far from theFinca Vigía, is a place to see where Hemingway kept his fishing boat and drank rum after returning from the sea. The fishing village itself remains virtually unchanged in the last 50 years and this is the best vantage point to explore one important aspect of Hemingway greatest novel, The Old Man and the Sea. La Tarraza, the novelest's favorite bar in Cojimar, is still in business.
For Hemingway, the sea occupied his time and his imagination and Cojimar was his main assess point. He was an avid fisherman and explorer of Cuba's vast coastline. On an artistic level, the sea represented both beauty and treachery. And in Hemingway's world, the capriciousness of the sea parallels the randomness of life and requires man to demonstrate his worthiness through noble action. Adherence to this stoic belief underpins one of Hemingway's most iconic statements from the Old Man and the Sea, "A man can be destroyed but not defeated."
La Terraza, Cojimar, Cuba
On a symbolic level, the character Santiago navigates his small fishing boat on the sea of Hemingway's subconscious, making the sea the bearer of artistic creation. From the village of Cojimar, visitors can sit in Hemingway favorite bar and look out over the same vista that inspired his most important visions are about the nature of humanity.
Yoan Capote ©Yoan Capote. Courtesy of the artist and Jack Shainman Gallery, New York.
Cuban artist Yoan Capote comes close to capturing the mystical and imaginative quality of the sea described in Hemingway's novel. Capote's Isla and Panagre series of seascapes were created by attaching thousands of fish hooks to a board. The artists says, "These paintings are the interior sea of every human being, the psychological seascape that every person in Cuba has."
So before you go to Cuba, study The Old Man and the Sea and participate in Hemingway's Cuba before it fades away behind an ocean of tourists.
Yoan Capote, Isla (After Bocklin), Detail, 2016. Oil, nails and fish hooks on linen mounted on panel. 42 1/8 x 74 7/16 x 5 1/8 inches. ©Yoan Capote. Courtesy of the artist and Jack Shainman Gallery, New York.
Hemingway images courtesy of the JFK Library
How To Explore Cuba Through Ernest Hemingway's Most Famous Novel was last modified: August 12th, 2017 by Simons Chase
Posted by Simons Chase
Categories CULTURE, TRAVEL	Post navigation
Music and Spirituality Converged With Cuban Drummer Francisco Aguabell


text:
One hundred and Fifty-Six
blog post, Filmmaker Interviews	|
When you first meet writer/director Carl Seaton, the word solid comes to mind – both in presence and in essence. Dude is serious. An auteur rooted in his craft, and point of view, a Brother with a purpose. We took a moment to discuss his latest offering to our series with Turn Down For What.
Tell me about the concept for your second hard-hitting FSC episode, Turn Down For What?
The idea came up after I heard Mr. Stand your ground was going to do a celebrity boxing match where he would fight DMX. The fact that he is viewed as a celebrity didn't sit well with me so I started bouncing around various scenarios. I'm also a student of history and when I read about the historical reference I mentioned in the piece it affected me deeply on multiple levels.
This was also written and shot months before Ferguson occurred and the #blacklivesmatter movement began. In this social media age things get a lot of attention then everyone moves on to the next sensationalized topic. I didn't want the senseless murder of Trayvon to fade in to obscurity ever.
It was also done to reflect the frustration that a lot of men feel about the depiction black men and boys in the country. That is why I felt it would resonate when we find out there professions.
What would you say to those who find the subject matter controversial?
It is meant to be controversial as well as polarizing but more importantly I hope that it provokes discussion and gives insight in to the intellect, frustration, and sensitivity of African American men.
What is the most important thing for you as an artist, as a filmmaker? What inspires you?
The mantra that I live by is "Art is the lie that allows us to see the truth." As an artist the most important thing to me is telling authentic stories that elevate, inspire, and effect the audience. Great storytelling inspires me. A film like #whiplash which is a small film about a drummer is an amazing story with a great theme that inspired me recently. Game changing content inspires me as well but it's not just film or TV, it can be music, a photo, or even a quote.
When casting for your FSC episodes, what is your process for selecting the right actors? What do you look for?
I am fortunate to know so many extremely talented actors personally that I have seen in other films, web series, and in the theater. I look at a lot of things and make mental notes on people to watch. What I look for is an actors investment in the performance. I've seen very bad films with great performances by actors.
Some would say this is an opportune time for Black cinema and television, while others point to the lack of diversity at award shows as an indicator of how far we are from full representation. Is there anything in particular you would you like to see going forward in regards to african american filmmakers?
Yes diversity is a huge topic of discussion right now but more importantly the industry is shifting away from the the normal distribution outlets. The millennials are showing us all how to generate content while building Your audience that will follow you wherever you go. This ideology of self empowerment is the kind of thing that will shift the standard way of how things have gone before.
Any upcoming projects you can share with our viewers?
I'm currently working on two different features LINE, a dramedy about brotherhood and W4M, an erotic thriller.
blog post, Episodes	|
The Hate That Hate Made
A Black Man relives a racial wound from his past. Written and directed by Allen L. Sowelle, this episode features Thomas Anthony Jones and Oliver Macready.
allen l. sowelle, bullying, hatred, jim crow, n-word, oliver macready, prejudice, racial epithet, racism, thomas anthony jones, violence	| Comment
A father confronts his son about skipping school, only to be devastated upon learning why. With honest portrayals by Esquire Duke and Myron Maurice Parran, this episode was written by Cassandra Cooper, and directed by Allen L. Sowelle
achievement gap, allen l. sowelle, bigotry, bullying, cassandra cooper, divorce, esquire duke, fsc, myron parran, parenting, prejudice, racial inequality	| Comment


text:
Conrado I el Viejo (nacido alrededor de 800 - murió el 22 de marzo o 21 de septiembre, entre 862 y 866), era hijo de Guelfo I, señor de Baviera, y Heilwig de Sajonia (c.775-833). Fue conde de Argengau, de París y de Auxerre, y fue abad laico de Abadía de San Germán de Auxerre y de la Abadía de San Galo.

Biografía
Fue uno de los primeros Güelfos, miembro de la rama bávara, era hermano de Judith la segunda esposa de Luis el Piadoso y de Emma de Altdorf, esposa de Luis el Germánico. Debido a esta relación, se convirtió en uno de los confidentes más cercanos del emperador Luis el Piadoso, el también compartió su detención entre 833/834,
y logró construir una fuerte posición de poder de su familia en la Alta Suabia.

Fue uno de los tres embajadores, que fueron enviados en 842 por Carlos el Calvo y Luis el Germánico, con poderes extraordinarios donde su hermano Lotario I, para negociar la división del reino de los francos y ofrecer a Lotario la tercera parte del imperio designada por ellos. Los otros dos enviados fueron la Adalard el Senescal y el conde Cobbo el Viejo.

Casado con Adelaida de Alsacia también llamada Adelaida de Tours (c.805 - después de 866), hija de Hugo de Tours, el Cobarde († 837), conde de Sundgau y Tours, tuvo los siguientes hijos:
 Guelfo II († 858);
 Hugo (c. 830 - † 886) abad de San Germán de Auxerre, Noirmoutier, San Ricario, San Bertin y San Martín de Tours, arzobispo de Colonia entre 864-870;
 Conrado II de Borgoña († 876), conde de Auxerre, duque de Borgoña;
 Judith, la esposa de Udo de Neustria;
 Rodolfo, † antes de 864, abad de San Ricario, 849 abad de Jumièges

En 858, durante una campaña en el reino de los francos del oeste, él y su familia abandonaron a su soberano Luis el Germánico y se acercaron a Carlos el Calvo, hijo de Judith. Ellos fueron generosamente recompensados y a Conrado se le designó otros condados. Luis el Germánico confiscó sus feudos y tierras bávaras.

El Miracula Sancti Germani llama a Conrado, princeps Chuonradus (príncipe, soberano), al registrar su matrimonio. Según algunas versiones su esposa se volvió a casar con Roberto el Fuerte después de su muerte.

Títulos nobiliarios
Konrad llevó a los siguientes títulos:
 830 -"Dux nobilissimus" (es decir, duque en Alemania)
 839 - después de 849- Conde en Argengau,
 839 - Conde en Alpgau,
 844 - Conde en Linzgau,
 849 -Conde de París,
 después de 860 conde de Auxerre como seguidor de Carlos el Calvo;

Referencias

Bibliografía
 Johann Gottfried Eichhorn: Urgeschichte des erlauchten Hauses der Welfen, Brüder Hahn, Hannover 1816.
 Bernd Schneidmüller: Die Welfen: Herrschaft und Erinnerung (819-1252), Kohlhammer, Stuttgart 2000, ISBN 3-17-014999-7

Enlaces externos

Duques de Borgoña del siglo IX
Condes de París
Condes de Francia
Señores de Francia
Nobles de Francia del siglo IX
Abades

redpajama_set_name: RedPajamaWikipedia