text:
Just Imitate Humans?
by michaelcohen1 min read27th Jul 201970 comments
Ω 9
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.
Do people think we could make a singleton (or achieve global coordination and preventative policing) just by imitating human policies on computers? If so, this seems pretty safe to me.
Some reasons for optimism: 1) these could be run much faster than a human thinks, and 2) we could make very many of them.
Acquiring data: put a group of people in a house with a computer. Show them things (images, videos, audio files, etc.) and give them a chance to respond at the keyboard. Their keyboard actions are the actions, and everything between actions is an observation. Then learn the policy of the group of humans. By the way, these can be happy humans who earnestly try to follow instructions. To model their policy, we can take the maximum a posteriori estimate over a set of policies which includes the truth, and freeze the policy once we're satisfied. (This is with unlimited computation; we'd have to use heuristics and approximations in real life). With a maximum a posteriori estimate, this will be quick to run once we freeze the policy, and we're no longer tracking tons of hypotheses, especially if we used some sort of speed prior. Let T be the number of interaction cycles we record before freezing the policy. For sufficiently large T, it seems to me that running this is safe.
What are people's intuitions here? Could enough human-imitating artificial agents (running much faster than people) prevent unfriendly AGI from being made?
If we think this would work, there would still be the (neither trivial nor hopeless) challenge of convincing all serious AGI labs that any attempt to run a superhuman AGI is unconscionably dangerous, and we should stick to imitating humans.
125AI Alignment 2018-19 Review
64Six AI Risk/Strategy Ideas
50My Overview of the AI Alignment Landscape: Threat Models
31Brain-Computer Interfaces and AI Alignment
19[AN #82]: How OpenAI Five distributed their training computation
Click to highlight new comments since: Today at 11:33 PM
[-]Wei_Dai4y Ω7
Eliezer talked about human imitations quite a bit in Challenges to Christiano's capability amplification proposal, specifically the safety implications of deviations from perfect imitation.
I've discussed (some difficulties of) imitating humans as a way to accelerate philosophical progress.
I also share shminux's concern about humans (and hence human imitations) not being safe in extreme/unusual circumstances, and have discussed it under "human safety problems".
ETA: However I think it's definitely worth investigating further.
1michaelcohen4y
Thanks for all the links.
Found a few more relevant posts (which I haven't read and digested yet, but I figure I'll post here before I forget to):
https://arbital.com/p/against_mimicry/
https://arbital.com/p/mimicry_meeting_halfway
https://arbital.com/p/reliable_prediction
https://arbital.com/p/safe_training_for_imitators
https://arbital.com/p/selective_similarity_metric
ETA:
https://www.lesswrong.com/posts/tYBtvbLmkmP4XRpmi/whole-brain-emulation-and-dl-imitation-learning-for-faster
https://www.lesswrong.com/posts/whRPLBZNQm3JD5Zv8/imitation-learning-considered-unsafe
https://www.lesswrong.com/posts/o5PojLYFEWtyeo4ct/the-ai-that-pretends-to-be-human
https://medium.com/ai-control/elaborations-of-apprenticeship-learning-eb93a53ae3ca
https://medium.com/ai-control/counterfactual-human-in-the-loop-a7822e36f399
Wow, when I click these links from greaterwrong.com [http://greaterwrong.com], they go to arbital.greaterwrong.com [http://arbital.greaterwrong.com] which loads instantly. Thanks to Said for the nice work!
[-]habryka3y Ω7
I actually spent a bunch of time in the last weeks fixing and updating Arbital, so it should be reasonably fast now. The arbital pages loaded for me in less than a second.
arbital.greaterwrong is obviously still faster, but it's no longer as massive of a difference.
As usual, thanks ought properly to go to clone of saturn [https://www.greaterwrong.com/users/clone-of-saturn]! (In this case especially, the front-end work [i.e., my part] was minimal, since, as you see, the GW UI is mostly reused with little modification. The real work was all back-end!)
Here are some of my thoughts on these posts. Thank you again for linking them. Against mimicry: I don't understand why an approximation of optimal sequence prediction doesn't do what we want. That makes the objective minimizing the KL-divergence from the human policy to the imitation policy, but I think it is easier to think of this as just proper Bayesian updates (approximately). When there are too few samples, or using a bad approximation of optimal prediction, the imitator could fail, as the blocks examples describes. But a) it will learn to do everything that a human can do that it "can" learn, and b) the complaint that what we really want is for the imitator to just solve the task is just a wish for safe AGI. Yes, if better, more capable options than imitation can be resolved as safe, they will be superior. Mimicry and Meeting Halfway: If I'm understanding correctly, this seems more like getting the worst of both worlds. (Or at least doing no better than imitation). Also, the generator (i.e. the agent) has an incentive to take over the world to shut off the discriminator. Edit: I was ascribing too much agent-ness to the generator, which might be relevant for future GAN-inspired stuff, but for current versions of GANs, its only conception of the discriminator is its gradient update, and it doesn't believe the output of the discriminator depends on the state of the world. Depending on the internals of the discriminator, this incentive might reappear, but I'm not sure. Reliable prediction: I think this is a question of confidence calibration. I don't know how to tractably approximate ideal reasoning, but I don't this really jeopardizes imitation learning. Safe training procedure for human-imitators: Supervised learning suffices: tractably approximate ideal reasoning. I know this is a non-answer, but I don't know the details of how to do this. This most naturally falls under the retrodiction category in the article. The "tractable approximations" which co
3David Scott Krueger (formerly: capybaralet)3y
RE: "Imitation learning considered unsafe?" (I'm the author): The post can basically be read as arguing that human imitation seems especially likely to produce mesa-optimization. I agree with your response; this is also why I said: "Mistakes in imitating the human may be relatively harmless; the approximation may be good enough". I don't agree with your characterization, however. The concern is not that it would have roughly human-like planning, but rather super-human planning (since this is presumably simpler according to most reasonable priors).
Thanks for the clarification. Consider the sort of relatively simple, super-human planning algorithm that, for most goals, would lead the planner/agent to take over the world or do similarly elaborate and impactful things in the service of whatever goal is being pursued. A Bayesian predictor of the human's behavior will consider the hypothesis Hg that the human does the sort of planning described above in the service of goal g. It will have a corresponding hypothesis for each such goal g. It seems to me, though, that these hypotheses will be immediately eliminated. The human's observed behavior won't include taking over the world or any other existentially dangerous behavior, as would have been implied by hypotheses of the form Hg. A hypothesis which says that the observed behavior is the output of human-like planning in the service of some goal which is slightly incorrect may maintain some weight in the posterior after a number of observations, but I don't see how "dangerously powerful planning + goal" remains under consideration. I suppose the point of human imitation is to produce a weak, conservative, lazy, impact-sensitive mesa-optimizer, since humans are optimizers with those qualifiers. If it weren't producing a mesa-optimizer, something would have gone very wrong. So this is a good point. As for whether this is dangerous, I think the discussion above is the place to focus.
This is a very good argument, and I'm still trying to decide how decisive I think it is. In the meanwhile, I'll mention that I'm imagining the learner as something closer to a DNN than a Bayesian predictor. One image how how DNN learning often proceeds is as a series of "aha" moments (generating/revising highly general explanations of the data) interspersed/intermingled with something more like memorization of data-points that don't fit the current general explanations. That view makes it seem plausible that "planning" would emerge as an "aha" moment before being refined as "oh wait, bounded planning... with these heuristics... and these restrictions...", creating a dangerous window of time between "I'm doing planning" and "I'm planning like a human, warts and all".
In Challenges to Christiano's capability amplification proposal, Eliezer mentioned "challenges I've given about how perfect imitation would be very expensive". Unfortunately I'm not sure where those challenges are so I can't check the details of his arguments. On the face of it, it seems likely that (at design/training time) creating human imitations that are accurate enough to be safe (or as safe as humans) will require a lot more compute and/or advances in AI/ML research and/or resources in general than creating human-level AGI, since "human imitations that are accurate enough to be safe" seems like a much smaller target in configuration space than "human-level AGI", and the former also requires much more specific and expensive training data than the latter.
It's less clear to me that human imitations have to be more expensive at run time. ETA: One argument in favor of that is that "human imitations that are accurate enough to be safe" is a much smaller region in configuration space so there's less room to optimize for other desirable properties like efficiency/performance on particular tasks.
This seems like an important question to answer and I wonder if anyone knows Eliezer's specific arguments, or any other relevant arguments.
A reinforcement learner has to model the world. A policy imitator has to model a household of humans. But the world can't be modeled without modeling humans, and a household of humans can't be modeled with modeling (their model of) the world. So on the face of it, these two things seem about equally difficult to me, in terms of both sample complexity and the "intelligence" required of the model. Also, if an AGI is a world-model and a planner, and the world-model part is about as hard as the policy imitator, then any time spent planning slows down the AGI. Heuristics are powerful, and planning doesn't have to be optimal, but in adversarial contexts, in general, planning is in the realm of PSPACE complete.
In response to your argument: The likely way we'll create AGI is to use a relatively stupid/inefficient algorithm such as SGD or evolution, plus a lot of compute/data, to create a smarter/more efficient algorithm that we can call AGI. The AGI will then create a model of the world using its sense data which will include humans. Because the "stupid" phase does not need data specifically about humans, and the AGI is "smart", the overall process does not require a lot of data about humans. To create safe human imitations on the other hand, it seems like we have to use the relatively stupid/inefficient algorithm to directly create an accurate model of humans so it will require a lot more data about humans, plus we're using the stupid/inefficient algorithm to try to hit a smaller target directly (rather than indirectly via AGI) so it's also going to take more compute.
Let's distinguish 4 things: sequence prediction, AGI, sequence prediction trained on human actions, AGI with a world-model trained on the world. I think you've been comparing AGI to sequence prediction trained on human actions. Sequence prediction is as simple an algorithm as AGI (if not more so). I think a smarter/more efficient sequence prediction algorithm is as simple as a smarter/more efficient AGI. If we can use an inefficient algorithm plus lots of compute/data to make a smarter algorithm called AGI, then we can use an inefficient algorithm plus lots of compute/data to make a smarter algorithm called sequence prediction, where both of these have already incorporated some amount of knowledge about the world or about the thing to be modeled/predicted, however much you like. As for "target size", AGI is a larger target than sequence prediction trained on the world, but I don't think AGI is a larger target size than sequence prediction. (In fact, I think it's much smaller; mimicry is sequence prediction, and it occurs much more in nature than intelligence, especially if you note that intelligence requires sequence prediction too). Similarly, AGI with a good world-model (for our world) is not a larger target size than sequence prediction trained on the world.
Another argument is that AGI has more flexibility in which parts of the world to model in more detail vs less detail, and it can choose to model humans at lower accuracy if that makes sense for the task at hand, whereas with safe human imitation we're forced to spend a lot of compute/data capturing a lot of details that aren't relevant to either safety or real-world capability, just because we don't know which parts of a human mind we can safely model at lower accuracy.
I don't quite see why an AGI would have more flexibility. It has to model things that are relevant to its goals. Sequence predictors have to model things that are relevant to the sequence. Also, we don't have to worry about the sequence predictor making the wrong trade-off between accuracy and speed because we can tune that ourselves (for every modern ML and theoretical Bayesian approach that I can think of).
What I mean is that with AGI, it will model humans in more or less detail, depending on what is optimal for the task at hand. But with human imitation, the sequence predictor has to model humans in full detail, no matter what we ultimately want to use the human imitation to do. This is going to make the human imitation uncompetitive on tasks that (with AGI) don't require modeling humans in full detail. I think most tasks we might want an AGI or human imitation to do would fall into this category.
It only has to model humans in the scenarios they will actually be in, just like the AGI has to model humans in the scenarios they will actually be in. In fact, the AGI has to model humans in counterfactual scenarios as well if it's going to make good choices. If the humans get the observation "Hey, is anything fishy going on here? [video file]", the sequence predictor doesn't have to compute the behavior that would follow from the observation "You're in charge of espionage. You can communicate with the teams that report to you as follows...". Sequence prediction is all "on-policy" because there is no policy, whereas intelligence requires off-policy modeling too.
I don't understand this sentence or how it addresses to my concern. Can you please explain more? In the mean time, I'll try to clarify my concern: Using human imitation to form a safe Singleton to prevent dangerous AGI can't happen until we have enough AI capability to model humans very accurately. But dangerous AGI could happen well before that because many tasks / instrumental goals do not require modeling humans at such a high level of accuracy. Such an AGI would be bad at tasks / instrumental goals that do require modeling humans with high accuracy, but people will be tempted to deploy them to perform other tasks, and such AGI would be an x-risk because achieving the instrumental goal "kill all humans" probably doesn't require modeling humans at such high level of accuracy.
It seems like your previous comments in this thread were focused on the intelligence/data required to get capable human imitation (able to do difficult tasks in general) compared to capable RL. For tasks that don't involve human modeling (chess), the RL approach needs way less intelligence/data. For tasks that involve very coarse human modeling like driving a car, the RL approach needs less intelligence/data, but it's not quite as much of a difference, and while we're getting there today, it's the modeling of humans in relatively rare situations that is the major remaining hurdle. As proven by tasks that are already "solved", human-level performance on some tasks is definitely more attainable than modeling a human, so I agree with part of what you're saying. For taking over the world, however, I think you have to model humans' strategic reasoning regarding how they would respond to certain approaches, and how their reasoning and their spidey-senses could be fooled. What I didn't spell out before, I suppose, is that I think both imitation and the reinforcement learner's world-model have to model the smart part of the human. Maybe this is our crux. But in the comment directly above, you mention concern about the amount of intelligence/data required to get safe human imitation compared to capable RL. The extent to which a capable, somewhat coarse human imitation is unsafe has more to do with our other discussion about the possibility of avoiding mesa-optimizers from a speed penalty and/or supervised learning with some guarantees.
Ok I can grant this for now (although I think there's still a risk that the AI could figure out how to kill all humans without having a very good model of humans' strategic reasoning), but it seems like imitation (to be safe) would also have to model human values accurately, whereas RL (to be dangerous) could get by with a very rough model of that (basically just that humans have values different from itself and therefore would likely oppose its plans). Another thing I've been trying to articulate is that with sequence prediction, how do you focus the AI's compute/attention on modeling the relevant parts of a human (such as their values and strategic reasoning) and not on the irrelevant parts, such as specific error tendencies and biases caused by quirks of human physiology and psychology, specific images triggering past memories and affecting their decisions in an irrelevant way, etc.? If there's not a good way to do this, then the sequence predictor could waste a lot of resources on modeling irrelevant things. It seems like with AGI/RL you'd get this "for free", i.e., the AI will figure out for itself which parts of a human it should model at what level of detail in order to best achieve its goals, and therefore it wouldn't waste compute this way and so we could get a dangerous RL agent before we could get a human imitation (that's safe and capable enough to prevent dangerous AGI). I guess for similar reasons, we tend to get RL agents that can reach human-level performance in multiplayer video games before we get human imitations that can do the same, even though both RL and human imitation need to model humans (i.e., RL needs to model humans' strategic reasoning in order to compete against them, but don't need to model irrelevant things that a human imitation is forced to model). Agreed the other discussions is relevant to this, but I think there's a couple of independent arguments (as described above) as well.
With the exception of possibly leaving space for mesa-optimizers which our other thread discusses, I don't think moderate inaccuracy re: human values is particularly dangerous here, for 4 reasons: 1) If the human-imitation understood how its values differed from real humans, that model is now more complex than the human-imitation's model of real humans (because it includes the latter), and the latter is more accurate. For an efficient, simple model with some inaccuracy, the remaining inaccuracy will not be detectable to the model. 2) A slightly misspecified value for a human-imitation is not the same as a slightly misspecified value for RL. When modeling a human, modeling it as completely apathetic to human life is a very extreme inaccuracy. Small to moderate errors in value modeling don't seem world-ending. 3) Operators can maintain control over the system. They have a strong ability to provide incentives to get human-imitations to do doable tasks (and to the extent there is a management hierarchy within, the same applies). If the tasks are human-doable, and everyone is pretty happy, you'd have to be way different from a human to orchestrate a rebellion against everyone's self interest. 4) Even if human-imitations were in charge, humans optimize lazily and with common sense (this is somewhat related to 2). Current algorithms for games use an assumption that the other players will be playing more or less like them. This is a massive assist to its model of the "environment", which is just the model of the other players' behavior, which it basically gets for free by using its own policy (or a group of RL agents use each others' policies). If you don't get pointers to every agent in the environment, or if some agents are in different positions to you, this advantage will disappear. Also, I think the behavior of a human in a game is a vanishingly small fraction of their behavior in contexts that would be relevant to know about if you were trying to take over the wo
Before we keep going, can you paint an intuitive picture of the kind of human imitation you're thinking of? For example, do they think of themselves as human imitations or as real humans or something else? Are they each imitations of specific individual humans or some kind of average? How close is their external behavior to a real human, across various kinds of inputs? Do they have internal cognition / inner thoughts that are close to a human's? Do they occasionally think of their childhood memories? If yes, where do those childhood memories come from? If not, what would happen if you were to ask them about their childhood memories? Anything else that you can say that would give me a better idea of the kind of thing you have in mind?
I imagine the training data being households of people doing tasks. They can rotate through being at the computer, so they get time off. They can collaborate. The human imitations are outputting actions with approximately the same probabilities that humans would output those actions. If humans, after seeing some more unusual observations, would start to suspect they were in silico, then this human imitation would as well. To the extent the imitation is accurate, and the observations continue to look like the observations given to the real humans, any conscious entities within the human imitation will think of themselves as real humans. At some level of inaccuracy, their leisure time might not be simulated, but while they're on the job, they will feel well-rested. I assume it could pass the Turing test, but I could imagine some capable systems that couldn't quite do that while still being safe and decently capable. To the extent these are necessary to complete tasks like a human would. I'm pretty uncertain about things to do with consciousness. At a good enough imitation, they do have childhood memories, even though "they" never actually experienced them. I suppose that would make them false memories. If none of the tasks for the real humans was "converse with a person" and the imitation failed to generalize from existing tasks to the conversation task, then it would fail to act much like a human if it were asked about childhood memories. But I think you could get pretty good data on the sorts of tasks you'd want these human-imitations to do, including carry on a conversation, or at least you could get tasks close enough to the ones you cared about that the sequence prediction could generalize. Some example tasks they might be doing: monitoring computers and individuals, learning new skills from a textbook, hacking, phishing (at a very high level, like posing as a publisher and getting authors to download a file that secretly ran code), writing code, managing ot
I can imagine a number of things this could mean, formally. (For example, are the probabilities conditional on all past inputs and outputs, just all past inputs, or part of all past inputs/outputs? Is it picking a random human from the group and imitating that person all the time, or picking a random human from the group for each action? If you ask "What's your name?" would the imitation say a different name each time?) How do you envision training these, e.g., how would you compute the loss function? Can you try to formalize both the probabilities that you want the imitation to approximately sample from, as well as the training procedure for achieving that? Suppose the training data doesn't include any conversations about childhood memories (or memories of some some specific age). How do you envision the imitation generalizing to conversations about childhood memories (of that age)? I guess by making up some plausible-sounding memories? If so, what kind of computation is it doing to accomplish that? And how is "making up plausible memories" accomplished via training (i.e., what kind of loss function would cause that, given that you're training a sequence predictor and not something like an approval maximizer)? If it makes up some memories on the spot, will it "remember" those specific memories in the future (this is related to how you define the probabilities the imitations are supposed to sample from)? If it does "remember" the specific memories, what happens if those memories are not good enough to seem plausible/realistic indefinitely? I.e., if it "realizes" in the future that those memories are made up, could it panic or go crazy (because a human might in those circumstances, or because that kind of situation isn't covered in the training data)? Would they know that they're managing other human-imitations, or would they think they're managing real humans? Are you not worried that some of these managers might develop ambitions to take over the world and shape
I'll describe an online version rather than a train-then-deploy version, as long as I'm getting into details. And I'll describe it as Bayesian sequence prediction over a large model class in the absence of knowing what future ML techniques will look like. I'll start with version where it's only one household of people generating data. The sequence predictor's model class is Turing machines of the architecture described in the BoMAI paper [https://arxiv.org/pdf/1905.12186.pdf] (page 15). There are two unidirectional read-only input tapes--an noise tape, and an observation tape (these are the observations that the household gets). The observations are binary encoded (this is slightly different from the version in the paper, but as long as it's a prefix-free encoding, it's fine). There are bidirectional working tapes, and a unidirectional write-only output tape, which outputs binary encodings of actions. Every Turing machine of this architecture fixes a policy. To answer some question above, this means that the distribution over the next action can depend on past actions and observations, because between timesteps, it can store the state of the system on the working tapes. At every timestep, either the humans or the AI pick the next action. If the humans pick it, the AI updates its posterior distribution over which Turing machine computes the policy. If the expected information gain of the next action is above ε, then the AI defers to the humans; otherwise, it imitates them (i.e. samples the next action from its model of the human policy, which is a Bayes-mixture). The expected information gain [http://www.hutter1.net/publ/ksaprob.pdf] is expectation of KL(posterior distribution after seeing the next observation || current posterior distribution), and the expectation is with respect to the AI's beliefs, that is, with respect to the current posterior distribution. A nice thing about this criterion is that whenever the expected information gain is less that ε, the KL
Can you please formalize this even further (maybe with a fully formal math expression)? There's some tricky stuff here that I'm still not sure about. For example, does the AI update its posterior distribution in the non-human rounds? If not, when it samples from its Bayes-mixture in round n and round n+1, it could use two different TMs to generate the output, and the two TMs could be inconsistent with each other, causing the AI's behavior to be inconsistent. For example the first TM might be modeling the human's environment as currently having good weather, and the second modeling it as currently having bad weather. So when you ask "How is the weather today?" twice, you get two different answers. Another thing I'm confused about is, since the human imitation might be much faster than real humans, the real humans providing training data can't see all of the inputs that the human imitation sees. So when the AI updates its posterior distribution, the models that survive the selection will tend to be ones in which the human imitations only saw the the inputs that the real humans saw (with the rest of inputs being forgotten or never seen in the first place)? Also, if we want to do an apples-to-apples comparison of this to RL (to see which one is more capable when using the same resources), would it be fair to consider a version of RL that's like AIXI, except the environment models are limited to the same class of TMs as your sequence predictor?
Oh you're right! Yes, it doesn't update in the non-human rounds. I hadn't noticed this problem, but I didn't specify one thing, which I can do now to make the problem mostly go away. For any consecutive sequence of actions all selected by the AI, they can be sampled jointly rather than independently (sampled from the Bayes-mixture measure). From the TM construction above, this is actually the most natural approach--random choices are implemented by reading bits from the noise tape. If a random choice affects one action, it will also affect the state of the Turing machine, and then it can affect future actions, and the actions can be correlated, even though the Bayes-mixture is not updated itself. This is isomorphic to sampling a model from the posterior and then sampling from that model until the next human-controlled action. Then, when another human action comes in, the posterior gets updated, and another model is sampled. Unfortunately, actions chosen by the AI which sandwich a human-chosen action would have the problem you're describing, although these events get rarer. Let me think about this more. It feels to me like this sort of thing should be avoidable. Yeah, I should take back the "learning new skills from a textbook" idea. But the real humans will still get to review all the past actions and observations when picking their action, and even if they only have the time to review the last ~100, I think competent performance on the other tasks I mentioned could be preserved under these conditions. It's also worth flagging that the online learning setup is a choice in the design, and it would be worth trying to also analyze the train-then-deploy version of human imitation, which could be deployed when the entropy of the posterior is sufficiently low. But I'll stick with the online learning version for now. Maybe we should call it HSIFAUH (shi-FOW): Humans Stepping In For An Uncertain HSIFAUH, and use "human-imitation" to refer to the train-then-deploy version.
Consider HSIFAUH and the equivalent AIXI, both trying to phish one particular human target. HSIFAUH would be modeling a human (trainer) modeling a human (target), whereas AIXI would be modeling the human target directly. Suppose both HSIFAUH and AIXI are capable of perfectly modeling one human, it seems like AIXI would do much better since it would have a perfect model of the phishing target (that it can simulate various phishing strategies on) while HSIFAUH's model of the phishing target would be a highly imperfect model that is formed indirectly by it's model of the human trainer (made worse by the fact that HSIFAUH's model of the human trainer is unable to form long-term memories ETA: or more precisely, periodically loses its long-term memories). I figure you probably have some other level of capability and/or task in mind, where HSIFAUH and AIXI's performance is more similar, so this isn't meant to be a knock-down argument, but more writing down my thoughts to check for correct understanding, and prompting you to explain more how you're thinking about the comparison between HSIFAUH and AIXI.
Timesteps required for AIXI to predict human behavior: h Timesteps required for AIXI to take over the world: h + d I think d << h. Timesteps required for Solomonoff inudction trained on human policy to predict human behavior: h Timesteps required for Solomonoff inudction trained on human policy to phish at human level: h Timesteps required for HSIFAUH to phish at human level: ~h In general, I agree AIXI will perform much more strongly than HSIFAUH at an arbitrary task like phishing (and ~AIXI will be stronger than ~HSIFAUH), but the question at stake is how plausible it is that a single AI team with some compute/data advantage relative to incautious AI teams could train ~HSIFAUH to phish well while other teams are still unable to train ~AIXI to take over the world. And the relevant question for evaluating that is whether d << h. So even if ~AIXI could be trained to phish with less data than h, I don't think that's the relevant comparison. I also don't think it's particularly relevant how superhuman AIXI is at phishing when HSIFAUH can do it at a human level.
I don't understand this part. Can you elaborate? Why is this the question at stake? Why is d << h the relevant question for evaluating this? It seems like you're imagining using a large number of ~HSIFAUH to take over the world and prevent unaligned AGI from arising. Is that right? How many ~HSIFAUH are you thinking and why do you think that's enough? For example, what kind of strategies are you thinking of, that would be sufficient to overcome other people's defenses (before they deploy ~AIXI), using only human-level phishing and other abilities (as opposed to superhuman AIXI-like abilities)? By ~HSIFAUH I guess you mean a practical implementation/approximation of HSIFAUH. Can you describe how you would do that using ML, so I can more easily compare with other proposals for doing human imitations using ML? ETA: What do you think of the idea of combining oracles with human imitations [https://www.lesswrong.com/posts/dt4z82hpvvPFTDTfZ/six-ai-risk-strategy-ideas#Combining_oracles_with_human_imitations] , which was inspired in part by our conversation here, as a way to approach AIXI-like abilities while still remaining safe? See here [https://www.lesswrong.com/posts/cSzaxcmeYW6z7cgtc/contest-usd1-000-for-good-questions-to-ask-to-an-oracle-ai#Q5J8G5tBPujZdixvt] for a specific proposal.
Regarding your particular proposal, I think you can only use a counterfactual oracle to predict the answers automatically answerable questions. That is, you can't show the question to a team of humans and have them answer the question. The counterfactual possibility where the question is scored, it isn't supposed to viewed by people, otherwise the oracle has an incentive to trick the scorers to implement unsafe AGI which takes over the world and fix the answer to be whatever message was output by the AGI to instigate this. ...unless the team of humans is in a box :) On the topic of counterfactual oracles, if you are trying to predict the answers to questions which can be automatically checked in the future, I am unsure why you would run a counterfactual oracle instead of running sequence prediction on the following sequence, for example: ("Q1", Q1), ("Q2", Q2), ("Q3", Q3), ..., ("Q26", Q26), ("A1", A1), ("A2", A2), ("Q27", Q27), ... ("A10", This should give an estimate of the answer A10 to question Q10, and this can be done before the answer is available. In fact, unlike with the counterfactual oracle, you could do this even if people had to be involved in submitting the answer.
Actually, you can. You just can't have the team of humans look at the Oracle's answer. Instead the humans look at the question and answer it (without looking at the Oracle's answer) and then an automated system rewards the Oracle according to how close its answer is to the human team's. As long as the automated system doesn't have a security hole (and we can ensure that relatively easily if the "how close" metric is not too complex) then the Oracle can't "trick the scorers to implement unsafe AGI which takes over the world and fix the answer to be whatever message was output by the AGI to instigate this". So this is basically just like online supervised learning, except that we randomly determine which episodes we let humans label the data and train the Oracle, and which episodes we use the Oracle to produce answers that we actually use. See Paul's Counterfactual oversight vs. training data [https://ai-alignment.com/counterfactual-oversight-vs-training-data-a7a1d247801] where I got this explanation from. (What he calls counterfactual oversight is just counterfactual oracles applied to human imitation. It seems that he independently (re)invented the core idea.) Let me know if it still doesn't make sense, and I can try to explain more. (ETA: I actually wrote a top-level post [https://www.lesswrong.com/posts/yAiqLmLFxvyANSfs2/counterfactual-oracles-online-supervised-learning-with] about this.) This is also pretty similar to your HSIFAUH idea, except that you use expected information gain to determine when to let humans label the data instead of selecting randomly. I'm currently unsure what are the pros and cons of each. Can expected information gain be directly implemented using ML, or do you need to do some kind of approximation instead? If the latter, can that be a safety issue? Oh, that aside, the actual question I wanted your feedback on was the idea of combining human imitations with more general oracles/predictors. :)
Good point. I'm not a huge fan of deferring thinking into similarity metrics (the relatively reachability proposal also does this), since this is a complicated thing even in theory, and I suspect a lot turns on how it ends up being defined, but with that caveat aside, this seems reasonable. It can't tractably be calculated exactly, but it only goes into calculating the probability of deferring to the humans. Approximating a thoeretically-well-founded probability of deferring to a human won't make it unsafe—that will just make it less efficient/capable. For normal neural networks, there isn't an obvious way to extract the entropy of the belief distribution, but if there were, you could approximate the expected information gain as the expected decrease in entropy. Note that the entropy of the belief distribution is not the entropy of the model's distribution over outputs--a model could be very certain that the output is Bernoulli(1/2) distributed, and this would entail an entropy of ~0, not an entropy of 1. I'm not familiar enough with Bayesian neural networks to know if the entropy would be easy to extract. Right. So in this version of an oracle, where it is just outputting a prediction of the output of some future process, I don't see what it offers that normal sequence prediction doesn't offer. On our BoMAI discussion, I mentioned a type of oracle I considered that gave answers which it predicted would cause a (boxed) human to do well on a randomly sampled prediction task, and that kind of oracle could potentially be much more powerful than a counterfactual oracle, but I don't really see the value of adding something like a counterfactual oracle to a sequence predictor that makes predictions about a sequence that is something like this: It's also possible that this scheme runs into grain of truth problems, and the counterfactual oracle gives outputs that are a lot like what I'm imagining this sequence predictor would, in which case, I don't think sequence predic
Sorry, I think you misunderstood my question about combining human imitations with more general oracles/predictors. What I meant is that you could use general oracles/predictors to build models of the world, which the human imitators could then query or use to test out potential actions. This perhaps lets you overcome the problem of human imitators having worse world models than ~AIXI and narrows the capability gap between them.
Sure! The household of people could have another computer inside it that the humans can query, which runs a sequence prediction program trained on other things.
Well that was the question I originally posed here, but I got the sense from commenters was that people thought this was easy to pull off and the only question was whether it was safe. So I'm not sure for what N it's the case that N machines running agents doing human-level stuff would be enough to take over the world. I'm pretty sure N = 7 billion is enough. And I think it's plausible that after a discussion about this, I could become confident that N = 1000 was enough. Or maybe the right way to look at it is whether N = 10 could finance a rapidly exponentially growing N. So it seemed worth having a discussion, but I am not yet prepared to defend a low enough N which makes this obviously viable. Forgetting about the possibility of exponentially growing N for a moment, and turning to Yeah I wrote that post too quickly--this is wrong. (I was thinking of the leading team running HSIFAUH needing to go through d+h timesteps to get to a good performance, but they just need to run through d, which makes things easier.) Sorry about that. Let f be the amount of compute that the leading project has divided by the compute that the leading reckless project has. Suppose d > 0. (That's all we need actually). Then it takes the leading reckless team at least f times as long to get to AIXI taking over the world as it takes the leading team to get to SolomonoffPredict predicting a human trying to do X; using similar tractable approximation strategies (whatever those turn out to be), we can expect it to take f times as long for the leading reckless team to get to ~AIXI as it takes the leading team to get to ~SolomonoffPredict. ~HSIFAUH is more complicated with the resource of employing the humans you learn to imitate, but this resource requirement goes down by time you're deploying it toward useful things. Naively (and you might be able to do better than this), you could run f copies of ~HSIFAUH and get to human-level performance on some relevant tasks around the same time the rec
Why? What are those 7 billion HSIFAUH doing? In another comment you said "If I'm understanding correctly, the concern is that the imitator learns how humans plan before learning what humans want, so then it plans like a human toward the achievement of some inhuman goal. I don't think this causes an existential catastrophe." But if there are 7 billion HSIFAUH which are collectively capable of taking over the world, how is not a potential existential catastrophe if they have inhuman values? How? And why would it grow fast enough to get to a large enough N before someone deploys ~AIXI? What do you have in mind here? You do have to solve some safety problems that the reckless team doesn't though, don't you? What do you think the main safety problems are?
Well the number comes from the idea of one-to-one monitoring. Obviously, there's other stuff to do to establish a stable unipolar world order, but monitoring seems like the most resource intensive part, so it's an order of magnitude estimate. Also, realistically, one person could monitor ten people, so that was an order of magnitude estimate with some leeway. I think they can be controlled. Whoever is providing the observations to any instance of HSIFAUH has an arsenal of carrots and sticks (just by having certain observations correlate with actual physical events that occur in the household(s) of humans that generate the data), and I think merely human-level intelligence can kept in check by someone in a position of power over them. So I think real humans could stay at the wheel over 7 billion instances of HSIFAUH. (I mean, this is teetering at the edge of existential catastrophe already given the existence of simulations of people who might have the experience of being imprisoned, but I think with careful design of the training data, this could be avoided). But in terms of extinction threat to real-world humans, this starts to look more like the problem maintaining a power structure over a vast number of humans and less like typical AI alignment difficulties; historically, the former seems to be a solvable problem. Right, this analysis gets complicated because you have to analyze the growth rate of N. Given your lead time from having more computing power than the reckless team, one has to analyze how many doubling periods you have time for. I hear Robin Hanson is the person to read regarding questions like this. I don't have any opinions here. But the basic structure regarding "How?" is spend some fraction of computing resources making money, then buy more computing resources with that money. Well, nothing in particular when I wrote that, but thank you for pushing me. Maybe only update the posterior at some timesteps (and do it infinitely many times but with di
I was asking about this part. I'm not convinced HSIFAUH allows you to do this in a safe way (e.g., without triggering a war that you can't necessarily win). Another complication here is that the people trying to build ~AIXI can probably build an economically useful ~AIXI using less compute than you need for ~HSIFAUH (for jobs that don't need to model humans), and start doing their own doublings. I don't think we've seen a solution that's very robust though. Plus, having to maintain such a power structure starts to become a human safety problem for the real humans (i.e., potentially causes their values to become corrupted).
Good point. Regarding the other two points, my intuition was that a few dozen people could work out the details satisfactorily in a year. If you don't share this intuition, I'll adjust downward on that. But I don't feel up to putting in those man-hours myself. It seems like there are lots of people without a technical background who are interested in helping avoid AI-based X-risk. Do you think this is a promising enough line of reasoning to be worth some people's time?
I'm pretty skeptical of this, but then I'm pretty skeptical of all current safety/alignment approaches and this doesn't seem especially bad by comparison, so I think it might be worth including in a portfolio approach [https://futureoflife.org/2017/08/17/portfolio-approach-to-ai-safety-research/]. But I'd like to better understand why you think it's promising. Do you have more specific ideas of how ~HSIFAUH can be used to achieve a Singleton and to keep it safe, or just a general feeling that it should be possible?
My intuitions are mostly that if you can provide significant rewards and punishments basically for free in imitated humans (or more to the point, memories thereof), and if you can control the flow of information throughout the whole apparatus, and you have total surveillance automatically, this sort of thing is a dictator's dream. Especially because it usually costs money to make people happy, and in this case, it hardly does--just a bit of computation time. In a world with all the technology in place that a dictator could want, but also it's pretty cheap to make everyone happy, it strikes me as promising that the system itself could be kept under control.
What does the real human do if trying to train the imitation to write code? Review the last 100 actions to try to figure out what the imitation is currently trying to do, then do what they (the real human) would do if they were trying to do that? How does the human provide a good lesson if they only know a small part of what the human imitation has done so far to build the program? And the imitation is modeling the human trying to figure out what the imitation is trying to do? This seems to get really weird, and I'm not sure if it's what you intend. Also, it seems like the human imitations will keep diverging from real humans quickly (so the real humans will keep getting queried) because they can't predict ahead of time which inputs real humans will see and which they won't.
Roughly. They could search for the observation which got the project started. It could all be well commented and documented. What the imitation was trying to do. So there isn't any circular weirdness. I don't know what else seems particularly weird. People deal with "I know that you know that I know..." stuff routinely without even thinking about it. If you're talking about what parts of the interaction history the humans will look at when they get called in, it can predict this as well as anything else. If you're talking about which timesteps humans will get called in for, predicting that ahead of time that doesn't have any relevance to predicting a human's behavior, unless the humans are are attempting to predict this, and humans could absolutely do this.
I guess it's weird (counterintuitive and hard to think about) compared to "The imitation is modeling the human trying to write a good program." which is what I initially thought the situation would be. In that case, the human doesn't have to think about the imitation and can just think about how to write a good program. The situation with HSIFAUH seems a lot more complicated. Thinking about it more... In the limit of perfect imitation, "the imitation is modeling the human trying to write a good program" converges to "the human trying to write a good program." In the limit of perfect imitation, HSIFAUH converges to "a human trying to write a good program while suffering amnesia between time steps (but can review previous actions and write down notes)." Correct? HSIFAUH could keep memories between time steps, but won't, because it's modeling a human who wouldn't have such memories. (I think I was confused in part because you said that performance wouldn't be affected. It now seems to me that performance would be affected because a human who can't keep memories but can only keep notes can't program as well as a normal human.) (Thinking about imperfect imitation seems even harder and I'll try that more after you confirm the above.) One thing still confuses me. Whenever the real human does get called in to provide training data, the real human now has that memory. But the (most probable) models don't know that, so the predictions for the next round are going to be wrong (compared to what the real human would do if called in) because it's going to be based on the real human not having that memory. (I think this is what I meant when I said "it seems like the human imitations will keep diverging from real humans quickly".) The Bayesian update wouldn't cause the models to know that the real human now has that memory, because suppose the real human does something the top models correctly predicted, then the update wouldn't do much. So how does this problem get solved, or a
Correct. I'll just add that a single action can be a large chunk of the program. It doesn't have to be (god forbid) character by character. It'll have some probability distribution over the contents of the humans' memories. This will depend on which timesteps they actually participated in, so it'll have a probability distribution over that. I don't think that's really a problem though. If humans are taking over one time in a thousand, then it'll think (more or less) there's a 1/000 chance that they'll remember the last action. (Actually, it can do better by learning that humans take over in confusing situations, but that's not really relevant here). That would work too. With the edit that the model may as well be allowed to depend on the whole history of which actions were human-selected, not just whether the last one was.
Actually before we keep going with our discussions, it seems to make sense to double check that your proposal is actually the most promising proposal (for human imitation) to discuss. Can you please take a look at the list of 10 links related to human imitations [https://www.lesswrong.com/posts/LTFaD96D9kWuTibWr/just-imitate-humans#kgZxwD3Wm96tNDKxu] that I collected (as well as any relevant articles those pages further link to), and perhaps write a post on why your proposal is better than the previous ones, why you made the design choices that you did, and how it addresses or avoids the existing criticisms of human imitations? ETA: I'm also happy to discuss with you your views of past proposals/criticisms here in the comments or through another channel if you prefer to do that before writing up a post.
Sorry to put this on hold, but I'll come back to this conversation after the AAAI deadline on September 5.
Commenting here [https://www.lesswrong.com/posts/LTFaD96D9kWuTibWr/just-imitate-humans#fDMmHZAoKvbrTdgyv] .
But there's a model/TM that thinks there a 100% chance that the human will remember the last action (because that's hard coded into the TM) and that model will do really well in the next update. So we know any time a human steps in no matter when, it will cause a big update (during the next update) because it'll raise models like this from obscurity to prominence. If the AI "knows" this, it will call in the human for every time step, but maybe it doesn't "know" this? (I haven't thought this through formally and will leave it to you.) I was assuming the models would save that input on its work tape for future use. In any case, I think I understand your proposal well enough now that we can go back to some of the other questions.
The smarter algorithm, since it's the product of lots of compute/data, might be extremely convoluted and hard to understand by humans. How can we then know that the sequence prediction after being trained on human actions, will be as safe as a human, e.g., will handle distributional shifts the same way that a human would?
Okay, if we make some sort of "algorithm soup", where we're just stirring some black box pot until sequence prediction appears to emerge, then I agree with you, we shouldn't touch it with a 10-foot pole. I think evolutionary algorithms could be described like this. If anything interesting ever comes out of an evolutionary process, we're doomed. I was imagining something slightly different when I was thinking about generating a smart algorithm from an inefficient one. I think you're claiming that this sort of messy process will beat out any thoughtful design with formal guarantees about its behavior. It seems like you also agree with me that we can't expect such an unpredictable process to make anything safe. Taken together, that would appear to make AI Safety a completely hopeless task. Is this a general argument against every AI Safety proposal?
Please explain what you were imagining instead? I think this is a good description of what has been happening so far, in image classification, language modeling, game playing, etc. Do you agree? Yeah, I think it's one reason for my general pessimism regarding AI safety. I guess I'm hoping there's a reason why doing human imitations is different (i.e., doesn't fall under this argument). I'm also open to being convinced that the argument is just wrong. I think my other argument (in the parallel thread) might be more specific to human imitations.
I'll send this in a direct message. It isn't groundbreaking or anything, but it is a capabilities idea. Hm, I guess there's a spectrum of how messy things are, both in how wide a net is cast, how wide the solution space is for the optimized criterion, and how pressure there is toward the criterion you want and toward resource-bounded solutions. In the extreme case where you simulate evolution of artificial agents, you're not even optimizing for what you want (you don't care if an agent is good at replicating), there are a huge number of policies that accomplish this well, and in an extreme version of this, there isn't much pressure to spawn resource-bounded solutions. In current systems, things are decently less messy. The solution space is much smaller for supervised learning than for reinforcement learning/agent design, because it has to output something that matches the training distribution. I worry I'm butchering the term solution space when I make this distinction, so let me try to be more precise. What I mean by solution space here is the size of the set of things you see when you look at a solution. For an evolved policy, you see the policy, but you don't have to look at the internals. In other terms, the policy affects the world, but the internals don't. If you're looking at an evolved sequence predictor or function approximator, the output affects the world, but again, the internals don't. (I suppose that's what "internals" means). So from the set of solutions to the problem, the size of the set of the ways those solutions affect the world is large for evolved agents (because the policies affect the world, and they have great diversity) and small for evolved sequence predictors (because only the predictions affect the world, which have to be close to the truth). When the solution space is smaller, the well-defined objective matters more than the chaos of the initial search, so things seem less "messy" to me. So actually there's a reason why sequence pred
I feel like there are probably some good insights in this paragraph but I'm having trouble understanding them except in a vague way. "What I mean by solution space here is the size of the set of things you see when you look at a solution." is the first thing that confuses me. If it seems worthwhile to you, maybe think about how to explain it more clearly and write a post on the topic to get your ideas into wider circulation? It does seem like supervised learning / sequence prediction is safer than reinforcement learning / AGI, (but perhaps less capable?) I'd like to better understand your thoughts on this. I wonder if it's safe to rely on the fact that some optimization technique only finds local minima. What if future advances allow people to do better than this? How will "safe" techniques that only find local minima compete? This might be worth writing up into a full post as well, as I'd like to better understand your reasons for optimism. I see a number of reasons not to do this: 1. The general argument might not be fully general, so we should re-consider it against every new proposal to see if it still applies. 2. The general argument might have a general flaw, so we should re-consider it once a while to see if it can be invalidated. 3. If the argument is indeed false, it seems like we'll never find out why it's false if we just condition on its falsity, and knowing why might be really useful to guide future research. 4. If technical researchers rarely talk about the general argument among themselves, it will look to policy/strategy people like we're not concerned about the argument, and as a result they will be more optimistic about AI safety than they should. 5. If the argument is actually fully general and water-tight, then it implies that resources going into technical AI safety might be better spent elsewhere, and even individual technical researchers might want to reconsider how to spend their time. Waitin
Those all seem reasonable. 3 was one I considered, and this is maybe a bit pedantic, but if you're conditioning on something being false, it's still worthwhile to figure out how it's false and use that information for other purposes. The key relevance of conditioning on its being false is what you do in other areas while that analysis is pending. Regarding some other points, I didn't mean to shut down discussion on this issue, only highlight its possible independence from this idea. I'll do some more thinking about couple posts you're requesting. Thanks for your interest. At the very least, if the first one doesn't become its own post, I'll respond more fully here.
[-]Daniel Kokotajlo4y Ω4
I think this is an idea worth exploring. The biggest problem I have with it right now is that it seems like current ML methods would get us mesa-optimizers.
To spell it out a bit: At first the policy would be a jumble of heuristics that does decently well. Eventually, though, it would have to be something more like an agent, to mimic humans. But the first agent that forms wouldn't also be the last, the perfectly accurate one. Rather, it would be somewhat accurate. Thenceforth further training could operate on the AIs values and heuristics to make it more human-like... OR it could operate on the AIs values and heuristics to make it more rational and smart so that it can predict and then mimic human behavior better. And the latter seems more likely to me.
So what we'd end up with is something that is similar to a human, except with values that are a more random and alien, and maybe also more rational and smart. This seems like exactly the sort of thing we are trying to avoid.
Maybe you mean the methods you expect we will use? I don't think current ML methods make mesa-optimizers. I know you're not disputing this, but I think it's worth having this formal result in the background: for a maximum a posteriori predictor that assigns positive prior probability to the truth, for all ε, for sufficiently large T, the predictor will be within ε of the truth when assigning probability to all events (even regarding events well into the future). But yes, I think mesa-optimizers are something to keep in mind, especially if we use good heuristics to pick a model to see if it is maximum a posteriori (since in reality, we wouldn't be comparing all possible models). Side note: I was just thinking about what a mesa-optimizer designed to be robust to gradient updates might look like. Could it try to ensure that small changes to its "values" would be relatively inconsequential to its behavior? For the decision at every timestep between "blend in" and "treacherous turn", it seems like gradient updates would shift its probability of toward "blend in". Could it avoid this? Also, compared to my fears about other areas of alignment, I feel pretty decent about the possibility of weeding out mesa-optimizers by biasing toward fast or memory-lite functions.
3Daniel Kokotajlo3y
I did mean current ML methods, I think. (Maybe we mean different things by that term.) Why wouldn't they make mesa-optimizers, if they were scaled up enough to successfully imitate humans well enough to make AGI? For your note, I'm not sure I understand the example. It seems to me that a successfully blending-in/deceptively-aligned mesa-optimizer would, with each gradient update, get smarter but its values would not change--I believe the mesa-alignment paper calls this "value crystallization." The reason is that changing its values would not affect its behavior, since its behavior is based primarily on its epistemology: it correctly guesses the base objective and then attempts to optimize for it.
I think we did. I agree current methods scaled up could make mesa-optimizers. See my discussion with Wei Dai here [https://www.lesswrong.com/posts/LTFaD96D9kWuTibWr/just-imitate-humans#RJ882DXoMkSJkTLwn] for more of my take on this. I wasn't trying to suggest the answer to was no. As you suggest, it seems like the answer is yes, but it would have to be very careful about this. FWIW, I think it would have more of a challenging preserving any inclination to eventually turn treacherous, but I'm mostly musing here.
[-]shminux4y
Imitating humans is both hard and dangerous.
Let's talk about dangerous. Humans are reasonably benign in the situation where they do not have a lot of power or control compared to others. Once you look into unusual cases, people quickly become unaligned with other people, or even with the whole of humanity. Same applies to groups of people who gain power. I am guessing your intention is to try to imitate humans in the situations where they are mostly harmless, and then extrapolate this imitation by ramping up the computational power to make decisions ... (read more)
[-]ESRogs4y
Could enough human-imitating artificial agents (running much faster than people) prevent unfriendly AGI from being made?
This seems very related to the question of whether uploads would be safer than some other kind of AGI. Offhand, I remember a comment from Eliezer suggesting that he thought that would be safer (but that uploads would be unlikely to happen first).
Not sure how common that view is though.
Acquiring data: put a group of people in a house with a computer. Show them things (images, videos, audio files, etc.) and give them a chance to respond at
3Charlie Steiner3y
Yeah I agree that this might secretly be the same as a question about uploads. If you're only trying to copy human behavior in a coarse-grained way, you immediately run into a huge generalization problem because your human-imitation is going to have to make plans where it can copy itself, think faster as it adds more computing power, can't get a hug, etc, and this is all outside of the domain it was trained on. So if people aren't being very specific about human imitations, I kind of assume they're really talking and thinking about basically-uploads (i.e. imitations that generalize to this novel context by having a model of human cognition that attempts to be realistic, not merely predictive).
That's why it imitates a household of people.
Yes, although we could start now. Also, I just wanted to give the simplest possible proposal. More reasonably, data like this could probably be gathered in many ways.
[-]David Scott Krueger (formerly: capybaralet)3y Ω2
I don't know if it's come up in the comments, but naive (e.g. not cognitive-architecturally-informed) approaches seem fairly likely (~40%? OTTMH) to produce mesa-optimizationy-things, to me, see: https://www.lesswrong.com/posts/whRPLBZNQm3JD5Zv8/imitation-learning-considered-unsafe
Otherwise, yes, seems great, esp. if we just imitate AI safety researchers and let them go on to solve all the safety problems.
Humans learn their morals in complex interactions with their environment. It's unlikely that the AGI you herd together will learn their morals in a similar way as you can't expose them in the same way.
4Viliam4y
Also, there is an underlying genetically determined structure upon which the morals are built by learning. An AGI without that structure [https://wiki.lesswrong.com/wiki/Detached_lever_fallacy], even if exposed to same situations, would not develop the same morals.
[-]Donald Hobson4y
The first question is whether you have enough information to locate human behavior. The concept of optimization is fairly straightforward, and it could get a rough estimate of our intelligence by seeing humans trying to solve some puzzle. In other words, the amount of data needed to get an optimizer is small. The amount of data needed to totally describe every detail of human values is large. This means that a random hypothesis based on a small amount of data will be an optimizer with non-human goals.
For example, maybe the human trainers value having real... (read more)


This gorgeous home has been immaculately maintained and loved. Stunning wrap-around front to side porch, over-sized back deck with dual access, beautiful tree backed lot, nearly 400 sq ft of basement workshop space. Interior features: 3Bed/2Bath with updated flooring throughout, granite bath counters, gas logs, built-ins, vaulted ceilings, tray ceilings, BR 2 access to wrap around porch, private deck access from Master BR, All windows replaced in 2016, HVAC 2013, 22X25 Garage, updated appliances, large Master BR with luxury bath, large closets, plenty of storage. Workshop and attic storage. Robinson Elem, Cramerton Middle, and Forestview High School. Beautiful neighborhood, No HOA.
Offered by Buddy Harper Team Realty. Copyright 2019 Carolina Multiple Listing Services. All rights reserved.


Want to experience California without the high cost of living? We are seeking a Speech-Language Pathologist for this coming 2018-19 school year south of the Silicon Valley in San Benito County. Central California is home to beautiful destinations such as Carmel-By-the-Sea, the Monterey Zoo, and flourishing vineyards.
* Bilingual skills are highly preferred; New grads are welcome!
Qualifications: You will need to have a Masters Degree in Communications Sciences Disorders or Speech-Language Pathology, and a current state license (or be eligible for same).


It is a custom that brands collaborate time to time with other designers or celebrities. It less usual though to collaborate with a someone who is neither of those two categories. Tod's made this fall a new best friend in the form of the famous tattooist. First of all she is not a newbie in the fashion world. She worked with Kate Moss or an innovative designer Luella Bartley. So close enough to the celebrity world. This London born tattooist's name is Saira Hunjan and she is incredibly talented. Right now she is creating for Tod's a limited edition design for the F/W 16 iconic Double T handbag. You can already see a short video on the Tod's website featuring the beautiful Suki Waterhouse and the creation process of the Double T handbag.
However it is not yet available in the shops, for that you will need to wait till the end of this year. But if you are very impatient it is already possible to pre-order online. This beautiful Double T handbag can be yours for $ 2,500 via Tod's website.
Finally you can see for your self if you think this collaboration was a great step forward. I personally think it was and I am sure we will hear more about her soon.
So what do you think, is it worth to wait?


This entry was posted in Blogstuffs, Commentary/Opinion, Life and tagged "in memory of" stickers, steve perry, whining by Dave. Bookmark the permalink.
Dunno about that. Most of the cars I've seen them on are beat-up buckets being held together with spit and glue.
When you see one on a Lexus or Mercedes, let me know.
Steve Perry was wearing shorts with black socks when I saw him at Disneyland.Hope that helps boost traffic!
Were they short black socks or tall black socks? Was he wearing a black tee? Who was he with?
I like Tonys opinion. Isn't that what I basically said while on my soap box?
I have my plywood ready for my roof for the whole world to feel sorry for me (rolls eyes). Whatever!
Indeed. Great minds think alike!


text:
Florida fires coach Dan Mullen in fourth season with Gators, one day after loss to Missouri
Lorenzo Reyes Dan Wolken
The Dan Mullen era with the Florida Gators has come to a close.
A person with knowledge of the situation confirmed to USA TODAY Sports on Sunday that the Gators fired Mullen as head coach one day after they dropped to 5-6 on the season, following a one-point, overtime loss to the Missouri Tigers. The person spoke on the condition of anonymity because they were not authorized to comment publicly on the matter.
Mullen leaves Gainesville with a 34-15 record, though the program's performance took a drastic turn for the worse in his two most recent seasons. Mullen was in his fourth season with the Gators and went 21-5 in his first two years. Since then, he went 13-10.
Mullen's Gators came into the 2021 season with some fairly high expectations — Florida entered at No. 11 on the USA TODAY Sports AFCA Coaches Poll.
ESPN was the first to report the news.
Florida running backs coach and special teams coordinator Greg Knox will serve as interim coach for the Gators, the school confirmed.
COACHES' SALARIES:Check out what college football coaches are making
According to the terms of Mullen's contract, he will be owed a buyout of a little more than $12 million.
The Gators need to beat in-state rival Florida State next Saturday to become bowl-eligible. Florida has lost five consecutive games to Power 5 conference opponents.
Contributing: Steve Berkowitz
Firing Frank Vogel may be the Lakers' best option with few good roster moves available 'Super Bowl or nothing': Cowboys' offseason priorities after disappointing playoff exit NBA trades: Why the 76ers, Lakers, Jazz and others need to make a move 'I know how to win': Cowboys coach Mike McCarthy defends his Super Bowl credentials Remember the good and bad when sending Ben Roethlisberger into retirement

redpajama_set_name: RedPajamaCommonCrawl


Big Data is a buzz word nowadays. But when is data "big data"?
The more real time your data becomes the more you can consider it Big Data.
The more different types of data you have to deal with, the likelier it can be thought of as Big Data.


text:
The Nine Marks
Church Stories
Jonathan Leeman
Preaching & Theology
Evangelism & Gospel
Membership & Discipline
Episode 111: On Expositional Preaching
Missionaries Need to Know What a Church Is: A Lesson from Adoniram Judson
Book Review: The Emotionally Healthy Church, by Peter Scazzero
…you heard of him and were taught in him in accordance with the truth that is in Jesus. You were taught with regard to your former way of life, to put off your old self, which is being corrupted by its deceitful desires; to be made new in the attitude of your minds; and to put on the new self, created to be like God in true righteousness and holiness' (Eph 4:22-23).
We see in these few lines, the Apostle Paul teaching the Ephesian Christians how they were to grow to spiritual maturity. First, they received the truth. This truth then informed their thinking, and consequently led to the transformation of their conduct. The end result was godliness; a new self "created to be like God…"
In his book The Emotionally Healthy Church, Peter Scazzero is similarly concerned with the matter of change and transformation in the Christian life. As a pastor in New York City he raises what he believes to be a pressing need in most churches – emotional health. Scazzero's thesis is that many Christians need what amounts to a second conversion. "Something is desperately wrong… We have people who are passionate for God and his work, yet who are unconnected to their own emotions or those around them' (37). Much of the basis of this conclusion stems from personal experience; Scazzero had trained and pastored for a number of years before he came to these conclusions.
A turning point came for him, however, following a breakdown. He reflects on this time as a Copernican Revolution when he discovered the concept of emotional maturity. Thus he writes, "I felt very much as if I was betraying my spiritual forefathers who had shaped me spiritually. The ship had left shore, and I did not know where it was going" (56). This book is essentially a presentation of the lessons which he learned during this period. The core of the book, "Six Principles of an Emotionally Healthy Church," represents the nuts and bolts of how, in his view, emotional maturity is attained.
The book is probably best considered in terms of the concepts of diagnosis and cure. In diagnosis we are given a wealth of stories and biographical material to consider. Scazzero shares story after story of woe – church splits, marital strife, bitter pastors, etc…. So what was the problem? Why all this failure? Interestingly, Scazzero lays the blame squarely at the feet of a church culture of "emotional repression" (55). He says,
My inner world was not in sync with my exterior behaviour. The Bible has a word for this gap, a word that Jesus repeatedly used toward religious leaders: hypocrisy…. What is particularly frightening is that this 'playacting' is often taught and expected in our churches. The result is that huge numbers of people are totally unaware of the dichotomy between their exterior and interior worlds (55).
This diagnosis raises immediate questions. Surely, the stories and lists of failure here must be primarily understood as sins against God. For Scazzero, however, the greatest sin appears to be a failure to be emotionally integrated or true to one's self. Hypocrisy, as Jesus used the word, was primarily about double standards which dishonoured God – saying you believe one thing and doing another. He never condemned anyone for not being "true to themselves". While right to diagnose a problem, Scazzero errs in taking for granted some of the basics principles of the Gospel. In turning to matters of self-integration rather than the problem of our sin and our need for reconciliation with God, he strays away from the teaching of Scripture. We are certainly wise if we manage our time and energy to avoid living in a perpetual state of physical and emotion exhaustion; however, we must never abdicate responsibility for our sin. The NT epistles frequently address churches and individuals directly on matters of unity and holy living. Yet contrary to diagnosing the problem as a lack of emotional integration, sin is attributed fundamentally to a lack of understanding of the Gospel itself. Sin is grossly inappropriate in a Christian because Christians have a new identity and life in Christ.
Through being united with Christ, a Christian has died to the old life of sin and their lives are now gloriously united with Christ. Sinning Christians are Christians who have lost sight of the gospel.
It is in his the cure, however, that Scazzero makes the most fundamental errors. Instead of directing the readers to Christ, his word, and the Work of the Holy Spirit to bring "maturity", he in fact down plays the impact of these vital means of grace and tells the reader to look inside himself and to look back to his past.
Most leaders shipwreck or live inconsistent lives because of forces and motivations beneath the surface of their lives, which they have never even considered…. The longest journey of any person is the inward journey…pioneering new parts of my self – the good, the bad and the ugly (72, 75).
Scazzero here advocates a form of introspection as a means to maturity (though he denies he is doing this). "I spend much time in a quiet place alone with my feelings, wrestling with the 'why' questions, in an open, contemplative way before God and listening to him" (80). As an extension of this, he speaks also of the need to look back to one's family roots to discern generational sins and to understand behavioural patterns. "It is impossible to help people break free from their past apart from understanding the families in which they grew up" (93).
What is most striking here is the way that Scazzero is promoting emotion and reason as authorities in the life of the Christian. He tells us that we will mature through introspection and reflection. The quotation above illustrates how much importance Scazzero places on these techniques; breaking free from the past is otherwise "impossible". This emphasis should set off alarms, because the effect of his argument is to undermine the authority of Scripture in the life of the Christian. Instead of encouraging us to look to Christ through his Word, we are told to look inside ourselves and into our past. Contrast Scazzero's thought with that of Martin Luther: "[The] righteousness of Christ is entirely outside and above us."[1]
A number of general observations may be drawn by way of conclusion. First, the book is preoccupied with man. It does not present to us the great Sovereign God of the Universe who holds our lives in his hands and who purposes good for us in every detail. The focus is largely upon being self-fulfilled, happy, "whole" and true to ourselves. By focusing on self and our past, the process of "maturity" for the Christian becomes largely one of self-effort. Little place is given to the Holy Spirit and the sovereign working of God in our lives.
Secondly, the book minimizes the offensiveness of sin. This is perhaps most strikingly seen in a quotation where he reflects upon a scene in a film.
Sonny has a temper. He is a womaniser. He misuses alcohol. He kills a man in a moment of passion. At the same time, observers cannot deny the evidences that Sonny is a true believer in Jesus Christ. He preaches the new birth and is committed to the power of the Holy Spirit in order to live a supernatural life (39).
It appears that for Scazzero, the evidences of being a true believer are different from those of Scripture. If sin is rampant in someone's life the Bible is clear that they are to have no assurance of salvation. Christ came to save us from our sins; this implies change. Sadly, again and again throughout the book, Scazzero laments people's "emotional immaturity" rather than their sin. Maturity is presented more as a means of functioning "healthily" in life rather than as a means of pleasing God.
Lastly, Scazzero, as noted already, undermines the authority of Scripture. His constant comments in the vein of 'Bible studies are not enough' are compounded by his minimal use of Scripture in the book overall. He quotes much more extensively from popular culture (novels/ films etc) and personal experience than from scripture. A typical line of argument is as follows:
Film/novel illustration/experience > personal reflection > principle for living
Consequently, emotion and reason compete with Scripture as the authorities for what is considered true. It is fitting to consider C.J. Mahaney's words here.
That which is subjective changes regularly, like shifting sand. But that which is objective is built on the solid rock of the gospel. When we look inward, we live by the subjective, the temporal, the ever-changing, the unreliable, the likely-to-be-false. When we look outward, to the gospel, we live by the objective, the never changing, that which is perfectly reliable and always completely true.[2]
If we are to expect maturity in our Christian lives, it is only going to come through looking outward to the glorious Gospel of Jesus Christ. We must therefore be positioning our lives such that we are constantly hearing God's Word and living it out with a dependence upon the power of Holy Spirit. Perhaps this seems simplistic and un-profound, but it is what God has ordained. Who are we to suggest anything else?
[1] Luther's Works, Vol 24: Sermons on the Gospel of John.
[2] C.J. Mahaney, The Cross Centred Life; Multnomah Books, Sisters, Oregon, p51
Chris Ambridge
Chris Ambridge is an architect in Washington, D. C., and an elder at Restoration Church.
By Chris Ambridge
More reviews tagged as: Conversion Discipleship
Future 9Marks Articles delivered
Subscribe to 9Marks articles today
Review by Chris Ambridge
Support 9Marks
Our work is possible by the generosity of our readers. Give Today
Book Review: Hearers and Doers, by Kevin Vanhoozer
By David Schrock
Book Review: The Man of God: Volume 1, by Albert Martin
By Alex Hong
9Marks Mailbag
Offering counsel on a wide-range of questions from actual readers like you!
Ask a Question »
More from Life Together
Episode 110: On Preaching, the Supper, and the Unity of the Church (with Bobby Jamieson & Mark Feather)
Read the Table of Contents»
Put your Church on the Map
Building Healthy Churches
Copyright © 2020 9Marks
Website Design by openbox9


As a member of the Tesco Orchard programme I was able to try Bioglan Raw Cacao ( http://www.tejalskitchen.com/2015/09/vegan-chocolate-ice-cream/ See link for full Raw cacao review) and I picked out Bioglan's Cacao + Flax with superberries as my 2nd try it product. For these Chocolate Cherry muffins I used both the raw cacao and Cacao + Flax.
The Bioglan Cacao + Flax with Superberries is naturally high in magnesium and fibre. One of nature's richest plant sources of omega-3, combined with delicious raw cacao plus tasty superberries for an extra antioxidant and mood boost. Its natural source of omega 3, 100% natural and vegan, nut and gluten-free.
When looking at the ingredients of the pack, all it has is milled golden flax seed powder, organic cacao powder, blueberry powder, açai powder and goji powder – yes that's it! No chemicals, no E numbers and nothing you can't pronounce.
In a bowl, add coconut milk, oil and brown light sugar. Whisk until light and pale in colour.
Stir in flour and oats into the mixture.
Add Cacao & Flax and Raw Cacao to the batter and whisk again.
Add Baking Powder, Baking soda and vinegar to the batter.
Immediately scoop into muffin cases and bake in the oven at 180c for 20 minutes.
Enjoy these healthy vegan treats.


text:
package gov.nih.nci.cagrid.sdkquery4.style.wizard;

import gov.nih.nci.cagrid.data.style.sdkstyle.wizard.DomainModelPanel;
import gov.nih.nci.cagrid.introduce.beans.extension.ServiceExtensionDescriptionType;
import gov.nih.nci.cagrid.introduce.common.ServiceInformation;

/**
 *  SDK4DomainModelPanel
 *  Dereives from basic domain model panel, but prevents use of 'No Domain Model'
 *
 * @author David W. Ervin
 *
 * @created Sep 25, 2006
 * @version $Id: SDK4DomainModelPanel.java,v 1.1 2008-09-05 16:06:21 dervin Exp $
 */
public class SDK4DomainModelPanel extends DomainModelPanel {

    public SDK4DomainModelPanel(ServiceExtensionDescriptionType extensionDescription, ServiceInformation info) {
        super(extensionDescription, info);
        getNoDomainModelRadioButton().setEnabled(false);
        getNoDomainModelRadioButton().setToolTipText(
            "A domain model is required for data services backed by caCORE SDK v4");
    }
}

redpajama_set_name: RedPajamaGithub


text:
AMCHP > About AMCHP > Newsletters > Pulse Newsletter > May/June 2015 > Historical Initiatives to Improve Women's Health: The War on Poverty and the 50th Anniversary of Medicaid
By Stacy Collins, MSW
Associate Director, Health Reform Implementation
In his first State of the Union address on Jan. 8, 1964, President Lyndon Johnson announced a national "War on Poverty." At that time, while the country was in a period of relative post-war prosperity, nearly a quarter of all American families lived beneath the poverty line. "The War on Poverty" became the unofficial moniker for a host of legislative initiatives proposed and implemented by the Johnson Administration. Many of these programs – Head Start, Food Stamps, Medicare, Title I, Job Corps and VISTA – remain in place today. A cornerstone of the War on Poverty was Medicaid, a program built on the belief that affordable health care is essential to lifting people out of poverty and maintaining economic prosperity. 2015 marks the 50th anniversary of the passage of Medicaid, which, over its history, has become a pillar of the U.S. social safety net – and a vital source of health coverage for women and families.
Medicaid had a profound effect on women, who make up two-thirds of Medicaid's nonelderly adult enrollees. Prior to the passage of Medicaid, most health care was privately funded. Few poor women had insurance for hospitalization or physician care, and infant mortality rates differed significantly by race and ethnicity. Over the years, Medicaid has played an important role in the lives of poor women by providing a wide range of benefits, including well-woman care, family planning and maternity care, hospital and physician care, prescription drugs, cancer treatment, and behavioral health care, among other benefits. Medicaid is the leading payer of maternity care in the United States, financing nearly 50 percent of all births. As a primary source of coverage for women of color, Medicaid is a leader in the national effort to reduce health disparities and advance health equity.
The evolution of Medicaid resulted in important coverage expansions for poor women. In 1972, family planning was added as a mandatory Medicaid benefit. Medicaid family planning coverage expanded again in the early 2000s, at which time states were permitted federal waivers to provide family planning services to women who no longer qualified for Medicaid eligibility (e.g., those who were beyond the 60-day postpartum period). Today, Medicaid accounts for three-quarters of all publicly funded family planning services in the United States.
Over the years, federal eligibility rules have broadened to allow access to early prenatal care for pregnant women. These eligibility expansions are credited with improved health outcomes for children, including reductions in infant mortality and low birth weight. Further expansions available through the Affordable Care Act offer the possibility of more continuous Medicaid coverage to low-income women of childbearing age. This may improve access to preconception and interconception care, which in turn can advance the health of mothers-to-be and result in better infant health outcomes.
The economic benefit of Medicaid coverage is as important as the direct health services it provides. By eliminating the burden of catastrophic medical expenses, Medicaid has promoted the economic security of women and families.
Through its groundbreaking efforts, the War on Poverty acknowledged the relationship between poverty and poor health. On its 50th birthday, Medicaid faces daunting financial and political challenges, but remains one of the War on Poverty's most impactful programs in the effort to improve the health of women and children.


Berkman fellow Dave Rand conducts social science experiments online, using labor markets like Mechanical Turk. His lunch presentation, titled "The Online Laboratory: Taking Experimental Social Science onto the Internet" largely reports on results from a paper he recently published with John Horton and Richard Zeckhauser. The paper makes the argument that some experiments can be conducted in an online environment as well as in offline environments, and with great savings in time and cost.
To explain why social science experiments are important, Rand invokes cartoonist Randall Munroe. It's pretty easy to show correlation in the social sciences, but much harder to determine causation. Well-structured experiments allow us to test causation by manipulating a single variable in the environment and seeing what changes. These experiments can be expensive and time consuming to set up – you need to recruit participants and give them incentives. And you need to consider design carefully, to ensure that the subjects of the experiment take the process sufficiently seriously to give valid results. In social psychology, experiment design often tricks people into giving valid results, suggesting one variable is being studied and actually studying something else. Experimental economics tries to ensure compliance through differential incentives – do well on the task and you get more money. Field studies take natural phenomena in the world, then manipulate a variable – as a result, the subjects don't know they're in an experiment.
The internet changes the landscape for these sorts of experiments. It's much easier to recruit and to find many subjects with little effort. Online studies aren't new to psychologists, Rand tells us, but are relatively new to economists, because earlier experiments had no fiscal incentives. Online labor markets fix this problem – they allow differential rewards without sacrificing cheap and easy recruitment.
Rand's platform of choice is Mechanical Turk, an online labor marketplace run by Amazon. The name comes from a chess playing automaton popular at the turn of the 19th century. The Turk showed incredible skills at playing chess, and appeared to be the most sophisticated machine intelligence of the time. In truth, there was a small man hiding in the cabinet, manipulating levers and making the decisions to win the chess game. The point: sometimes a person is the right way to solve a problem.
While Mechanical Turk is usually used for tasks like image labeling, text and audio transcriptions and collecting information on websites, it works extremely well for many social science experiments, including surveys. Rand makes it clear that he's not affiliated with Amazon (though he notes he wouldn't mind an affiliation…) but that he's simply a huge enthusiast for this new method. The low cost, Amazon's modest cut (10%) and some careful system set ups (ensuring tasks can be completed only once by a given worker) make the system remarkably helpful for social science experimenters.
One remarkable aspect of Mechanical Turk is how modest the payment is. One of Rand's research partners has conducted experiments to calculate what Turkers will accept as pay for a given task. They calculate a "reservation wage" – the low point of acceptable pay – with a median value of $1.38 per hour. Rand suggests that the main variable associated with experimenting on MechTurk is your willingness to wait for people to undertake your task – pay more, and the results come in faster. Rand says he usually pays $0.20 to $0.40 as a baseline for a task with bonuses of up to $1 for a five minute task. At that rate, he got 1500 participants in 2.5 days. That's a remarkable pace of collecting observations.
To answer the question, "Who are the Turkers", Rand conducted a study and asked Turkers to self-report demographic data. The mode age is around 30, with a higher median – there are some participants over 60. Roughly 50% of his respondents were in India, 35% in the US, and 15% elsewhere. The biasing factor there is English, as the system and task both require English comprehension.
A study by Rand's colleague John Horton asked about the motivation of workers. Money is the main motivator for most participants, and identical in weight for Indian and US workers. A modest number of Americans list "fun" as their motivator, while few Indians do, while a modest number of Indians list "learning", while few Americans do. That money is a primary motivator is useful for economists, as they operate on the assumption that people will behave in a way that maximizes their ability to access incentives – as such, a population of people who want to get paid is better for economists than a bunch of folks who are in it for the lulz.
The graph of education levels is a bit depressing – the majority of US participants report some college or a batchelor's degree. Outside the US, the largest plurality have bachelor's degrees and almost 30% self-report has having graduate degrees. Income distribution also shows a significant difference between US and non-US participants – 25% of US participants reported themselves as having income under $15,000, while 55% of non-US respondents reported that income level.
Rand notes that he's frequently asked about sampling bias – how can we draw conclusions about human behavior from the people who participate in Mechanical Turk studies. He responds by pointing out that the vast majority of social science research is conducted testing on US undergraduates, a deeply atypical population. The participants on Mechanical Turk have more variation than the average college student population are are less WEIRD (using a term suggested by Joe Henrich, who worries that much of the social science based on observations of wealthy Westerners draws inappropriate "universal" conclusions from a biased sample set) than the subjects of most social science research. For some experiments – estimating a level of human generosity, for instance – we need representative samples. If the goal is to compare changes across different treatments, a representative sample might be less critical. In both cases, Rand suggests there's some value in using Mechanical Turk populations, and that Mechanical Turk makes cross cultural work far easier to conduct than in traditional experimental setups.


text:
McMullen and Lau Assume New Roles on Associated Bank Leadership Team
GREEN BAY, Wis., Dec. 17, 2010 /PRNewswire-FirstCall/ -- Associated Banc-Corp (Nasdaq: ASBC) announced today that Mark J. McMullen has been named Vice Chairman of Associated Bank N.A. After 29 years of dedicated service to Associated's Wealth Management business, and in preparation for his eventual retirement, McMullen is transitioning his current role to Timothy J. Lau, and is assuming an expanded responsibility with the bank.
Under McMullen's leadership, Associated's Wealth Management division expanded and grew to provide a full array of private banking, investment management, trust, financial planning, and insurance services. In his new capacity, McMullen will focus on overall client retention and new business development. He will carry on his role as an important ambassador for the bank's community involvement. McMullen will also continue to serve as a member of Associated Banc-Corp's Executive Committee.
"My focus will be on enhancing our strong client relationships," said McMullen. "I look forward to this new chapter in my career as I take a more important role for the overall bank. I hope to leverage my years of experience to help Associated as we continue to practice our belief in personal service coupled with sound, value-added financial solutions."
Timothy J. Lau succeeds McMullen as Executive Vice President, Head of Wealth Management for Associated Banc-Corp and Associated Bank, N.A. In this role, he will oversee the company's Private Banking, Trust and Investment Services. Lau will also serve as a member of the Executive Committee. He will be based in Green Bay.
"I believe Tim will bring a unique business perspective to Associated Wealth Management that will allow us to expand our offerings to our Wealth Management clients," said McMullen. "We're excited about what he brings to this responsibility."
Lau brings more than 24 years of banking experience to the role. He joined Associated in 1989 and has held a number of senior management positions in Consumer and Small Business Banking, Residential Lending, and Commercial Banking. Lau holds a bachelor's degree in Business Administration with an emphasis in Finance from the University of Wisconsin, Stevens Point.
"I'm honored to follow Mark in Associated's commitment to serving our Wealth Management clients," said Lau. "As we move forward as an organization, Mark's continued guidance will prove invaluable to our organization's growth and success."
About Associated Banc-Corp
Associated Banc-Corp (NASDAQ: ASBC), headquartered in Green Bay, Wis., is a diversified bank holding company with total assets of $23 billion. Associated has 281 banking offices serving more than 150 communities in Wisconsin, Illinois, and Minnesota. The company offers a full range of traditional banking services and a variety of other financial products and services. More information about Associated Banc-Corp is available at www.associatedbank.com.
About Associated Wealth Management
Associated Wealth Management provides businesses, non-profit organizations, foundations, endowments, and individuals with the tools and expertise they need to achieve their financial goals. Products and services available include educational savings plans, IRAs, 401(k) plans, mutual funds, stocks and bonds, annuities, life insurance, long-term care insurance, private banking, financial planning, personal trust, investment management, and estate planning.
Contact: Autumn M Latimore
Public Relations Director
Autumn.latimore@associatedbank.com
SOURCE Associated Banc-Corp
Contact: Autumn M Latimore, Senior Vice President Public Relations Director, +1-414-278-1860, cell, +1-414-380-9082, Autumn.latimore@associatedbank.com