text:
\section{Introduction}

The long term goal of creating agents capable of learning during extended periods of time, without constant supervision, will require the capability of continually learning about their environment.
Indeed, learning about the world is inherently a progressive/incremental task, since the data distribution experienced by the agent can change both spatially (e.g. countries) and temporally (e.g. seasons).
This involves building a model of an agent's surroundings, with visuals features, and continually updating this model as its life progresses and the environment evolves, without forgetting.

Recent advances in representation learning \cite{bengio2013representation} have brought successful tools for learning a model of the surroundings of the agent, a field known as State Representation Learning (SRL) \citep{lesort2018state}. Once trained, these representation models are used to learn policies using Reinforcement Learning (RL) more efficiently \cite{ha2018recurrent} than when using raw sensors. The process is presented in Fig.\ref{fig:srl}. However, previous approaches have mostly been limited to scenarios where the environment stays fixed. Since the models used are often neural networks trained using stochastic gradient descent (or any variant), they forget past knowledge when the training data distribution changes, an infamous problem called catastrophic forgetting \citep{french1999catastrophic}. In Continual SRL, the training data distribution changes as the environment evolves, which can happen in a discrete or continuous way. In our experiments, we consider sudden discrete changes of environment.

\begin{figure}[h!]
    \centering
    \includegraphics[scale=0.25]{figures/srl_2.png}
    \caption{The State Representation Learning paradigm is decoupled in two phases: first learn a representation model for collected sensory states $o_t$, then feed the state representations  $s_t$ as input to a policy $\pi_t = \pi(s_t)$, which is optimized to solve a task using RL. Previous work \citep{raffin2019decoupling}

    has shown this can be more efficient than using directly sensory states for RL.}
    \label{fig:srl}
\end{figure}

\begin{figure*}[t!]
    \centering
    \includegraphics[scale=0.38]{figures/S-TRIGGER.png}
    \caption{Overview of our proposed method S-TRIGGER for Continual State Representation Learning for RL. As the agent lives in environment 1, it collects sensory states (i.e. vision) to learn a state representation model (here, a VAE). We test this model by using it to solve, in a on-policy fashion, a RL task in environment 1. The agent is now moved to environment 2. Our method automatically detects this change, and use the VAE trained on environment 1 to generate images corresponding to environment 1, that are joint to collected sensory states of environment 2 to learn a state representation model for the two environments. We test this state representation model by learning, in a on-policy fashion, to efficiently solve a task in environment 1 \textbf{and} 2. The procedure continues as the agent encounters new environments. Best viewed in color.}
    \label{fig:s-trigger}
\end{figure*}

We propose a general method for Continual SRL for agents behaving in a changing environment, termed Self-TRIGgered GEnerative Replay (S-TRIGGER for short), with the aim of learning representations useful for RL. Our approach is compatible with any type of State Representation model that is generative and reconstructs the input, which includes Varational Autoencoders (VAEs) and its many variants (e.g. $\beta$-VAE \cite{higgins2016beta}, ACN \cite{graves2018associative} and others) that are widely used for SRL. Previous work have shown that VAEs can learn continually using generated samples from previous tasks, a method called Generative Replay \citep{lesort2018generative}. S-TRIGGER uses Generative Replay to \textbf{remember information relative to previously encountered environments}. With the goal of an autonomous agent in mind, we complement our approach with a general method for \textbf{automatic detection of environment change}, based on the reconstruction error distribution of VAEs. This detection method allows the VAE to self-trigger the Generative Replay when a new environment is encountered, without the need for the user to specify environment changes. Finally, our approach respects important Continual Learning (CL) desiderata: no access to past data and bounded system size. Fig.\ref{fig:s-trigger} presents an overview of S-TRIGGER.

We focus on settings where the agent has only access to sensory states (here, vision) that contain partial information about the environment, similarly to an embodied agent.
The agent should be able to use learned knowledge to facilitate policy learning with RL. Hence, we test our approach on a 2-D first-person simulator with coherent physics, in two scenarii where the environment changes over time. The first scenario is a proof of concept with 2 environments, with minimal change between the two. We then test the robustness of our approach in a more challenging setting: sequences of randomly generated mazes. We test whether the learned features provide efficient and high-performing RL training on a navigation task. We also measure the ability of our method to retain past knowledge, using reconstruction error and visualization. Our results show that our approach avoids catastrophic forgetting, provides features that enable efficient RL and detects environment changes almost perfectly.

\section{Related work}

We first review how generative models are used for SRL, then previous work related to Continual SRL for RL.

\subsection{State Representation Learning using generative models}

Generative models are commonly used for SRL, as they can successfully learn compressed and useful representations of visual images. When trained on an agent's sensory inputs, they provide a vectorial representation of what the agent experience. They have recently shown promising results as State Representation models for RL \citep{ha2018recurrent, wayne2018unsupervised}. They can also generate data, i.e. generate sensory states that the agent could have experienced.

Variational Auto-Encoders \citep{kingma2013auto} is a family of generative models that uses an encoder and a decoder. The encoder receives observable data $x$ as input and emits as output a data-conditional distribution $q(z|x)$ over latent vectors $z$. A sample $z \sim q$ is drawn from this distribution and used by the decoder to determine a $z$-conditional reconstruction distribution $p(x|z)$ over the original data. The loss function is composed of two terms: the expected negative log-likelihood of $x$ under $p$ (i.e. the reconstruction error), and a regularization term that constraints $q(z|x)$ to be close to some prior distribution $p(z)$. VAEs can then be used as generative models by sampling from $p(z)$ a code $z$ and decoding it. They are often used for SRL \cite{van2016stable, ha2018recurrent}, along with other types of auto-encoders, because the reconstruction error provide an intuitive way of analyzing what the model has learned in the latent representation. VAEs have the additional advantage of being able to generate data, which can be used for Continual Learning, as described thereafter.

\iffalse
This optimization problem can be written as the sum of two terms, which then can be used to derive a tractable lower bound objective to the true objective called the Evidence Lower Bound (ELBO):
\begin{equation}
\label{vaeloss}
\log (p_{\theta}(x|z)) \geq ELBO(\phi,\theta,x,z) = \mathbb{E}_{q_{\phi}(z|x)}[\log (p_{\theta}(x|z))] - D_{KL}(q_{\phi}(z|x)\|p(z))
\end{equation}
where $D_{KL}$ is the Kullback-Lieber (KL) Divergence between the true and approximate posterior, and $\phi, \theta$ respectively represent the weights of the encoder and decoder.
\fi

\subsection{Continual State Representation Learning for RL}

Generative models, e.g. VAEs, are widely used for SRL. Hence, one approach for Continual SRL is to apply a CL strategy developed for generative models. While most work in CL is focused on discriminative models, there is a recent interest in CL approaches for generative models \citep{achille2018life, nguyen2017variational}. Proposed methods often rely on using generated samples to avoid forgetting \citep{wu2018memory, lesort2018generative}. This technique, which we use in this paper, is termed Generative Replay. For instance it is used in the algorithm VASE \cite{achille2018life}, where the authors propose a representation model that can, similarly to S-TRIGGER, learn continually and detect changes in the data distribution. VASE dynamically allows spare representation capacity to account for newly acquired information. While their approach is promising, the learned features were not tested on a RL setting, for which our approach is specifically designed and tested. Previous work \citep{raffin2019decoupling} has demonstrated that learning state representation that perform well for policy learning is not straightforward. In our experiments, we demonstrate that S-TRIGGER can indeed continually learn features that allow fast and high-performing policy learning.

Other types of continual learning strategies for generative models could be investigated. We decided to select Generative Replay because it respects important desiderata for CL: bounded system size, no access to previous data. To the best of our knowledge, we found no previous work on applying these CL approaches to State Representation Learning for RL.

Another line of work on Continual SRL for RL is presented in DARLA \citep{higgins2017darla}, which circumvent the problem of catastrophic forgetting by learning disentangled representations with a specific VAE architecture.
Their approach learns factored features that are robust to minimal modifications of the factors of the environment, ours continually update features as environment changes are detected. They show that learning a policy in a source domain with their representation as input can lead to zero-shot transfer by using the same policy in a target domain (similar to the source domain). The idea of learning one state representation model that generalize to environments similar to previously encountered ones is important and compelling, but it is not clear how this model could handle heavy environment changes such as new objects appearing. In our experiments we shade light on the failure cases of this approach. However, the two approaches are compatible so any progress on one side complements the other.

Continual SRL is also related to Transfer Learning in a RL context \citep{taylor2009transfer}. Indeed, the idea of learning knowledge that can be general enough to be used to learn in new/previous environments or for new/previous tasks is a long standing challenge of RL research. This is for instance investigated with the options framework for RL \cite{perkins1999using}, where defining general enough options is shown to lead to greater performance when learning a set of tasks. Our work is thus part of this context.

\section{Continual State Representation Learning with Self-Triggered Generative Replay}

We now present our method for Continual State Representation Learning, S-TRIGGER, which is designed for settings where the environment changes discretely over time.

\subsection{Learning continually with Generative Replay}
\label{sec:GR}
The considered problem is twofold: \textit{build a State Representation model} which can \textit{continually learn}, i.e. without forgetting. We use VAEs' encoding property
to handle the State Representation and the generative ability of VAEs to generate states of previously seen environments to avoid forgetting, a technique known as Generative Replay \citep{shin2017continual}. In our case, we encode the sensory state received by the agent.

Once an environment change is detected, we generate states using the latent space of the VAE trained on previous environments, and add those generated samples to states collected in the new environment. Then, we train the same VAE on the joint data. This procedure is illustrated in Fig.\ref{fig:s-trigger}. The environment changes are detected automatically, as described thereafter.

This option does not re-use past data. Additionally, it is more scalable than a rehearsal method where samples from each previous environments would be maintained in memory, which amounts to scaling linearly in memory cost. Indeed, as new environments are encountered, we only need to maintain one VAE for all previously encountered environments, which makes Generative Replay a bounded system size solution in terms of memory cost.

In theory, the training cost should scale linearly in number of environment changes. However, in our experiments, we use a fixed number $n$ of generated samples for each environment change, and we consider up to two environment changes (i.e. sequence of three  environments). $n$ is of the same order of magnitude than the number of collected samples in the environment. Hence, our method is bounded in terms of training cost for low number of environment changes.

\subsection{Self-Trigger: Automatic environment change detection}
Since we aim at constructing an autonomous agent, we complement the proposed method with automatic detection of changes in the environment. Once an environment change is detected, Generative Replay is triggered, as described in Sec.\ref{sec:GR}, which is why we refer to our method as Self-TRIGgered GEnerative Replay (S-TRIGGER for short).

The detection method is based on statistical hypothesis testing on VAE reconstruction error distribution. The intuition is: \textit{if the VAE reconstruction error distribution suddenly changes, then the environment must have changed.}. To formalize this intuition, we use Welch's $t$-test \citep{welch1947generalization} to test the hypothesis $\mathcal{H}_0$ that two sets
$x_1, x_2$ of randomly sampled VAE reconstruction errors have equal mean. We choose this test over the standard $t$-test because we do not have reasons to assume that the two samples' variance are equal. The statistic $t$ is, under $\mathcal{H}_0$, distributed as a Student distribution with a number of degrees of freedom $\nu$ that can be approximated using the Welch-Satterthwaite equation:
\begin{equation}
    t=\frac{x_1 - x_2}{\sqrt{(s_{1}^{2}+s_{2}^{2})/N}}, \quad \nu \approx\frac{(N-1)(s_{1}^{2}+s_{2}^{2})^2}{(s_{1}^{4}+s_{2}^{4})},
\end{equation}
with N being the number of samples (assumed equal for both samples), and $s_1, s_2$ being the empirical standard deviations of samples 1 and 2, respectively. The decision of the test is based on a chosen significance level $\alpha$ for the test, by using the $p$-value. If the $p$-value is below $\alpha$, it suggests that the observed data is sufficiently inconsistent with $\mathcal{H}_0$ that $\mathcal{H}_0$ may be rejected and thus an environment change is detected.

Using the $p$-value on a statistical test is preferable for decision to using a threshold-based method directly on the error distribution. The threshold method is based on an arbitrary scale which depends on the considered sequence of environment. On the contrary, the test is a more general approach based on statistical principles and thus agnostic to scales. The $p$-value provides an interpretable parameter for controlling the detection method's recall, which is not possible with the threshold method. Additionally, we choose to use VAE reconstruction error distribution over the actual state distribution because we are interested in changes of the environment that require the VAE to be updated. For instance, if we add an already existing obstacle to the environment, the state distribution shifts, while the VAE reconstruction error distribution does not change because the VAE need not to be updated. The test is lightweight and can thus be continually performed.

\section{Experimental setting}

We present our experimental setting: environments, training methods and evaluation metrics.

\subsection{Environments}

Our environments are developed in Flatland \citep{caselles2018flatland}. Flatland is a lightweight 2D environment for fast prototyping and testing of RL agents. It is of lower complexity compared to similar 3D platforms (e.g. DeepMind Lab or VizDoom), but emulates physical properties of the real world, such as continuity, partially-observable states with first-person view and coherent physics. It is suitable for prototyping CL approaches as many experiments are required since several enviroments are used, which can quickly become computationally prohibitive on 3D platforms.

\subsection{Methods}

For the State Representation model, we experiment with a variant of $\beta$-VAEs \cite{higgins2016beta}. We emphasize that S-TRIGGER is general enough to be applied to any type of VAE. We train the $\beta$-VAE with a modified version of KL annealing \citep{bowman2015generating}. The original KL annealing method puts a weight of $0$ on the KL term and smoothly increase it to $1$. We found standard training (without KL annealing and with $\beta>1$) and this method ineffective on our environments: the models were either unable to reconstruct inputs properly or the generated samples were not satisfactory. Our version initializes training with a weight on the KL term of $1$, as in standard training, then smoothly reduces it until training is stopped when the reconstruction error does not improve. This inverse annealing scheme allowed successful training, as presented in Sec.\ref{sec:results} and Sec. \ref{sec:results2}. This training method is equivalent to CCI-VAE \cite{burgess2018understanding}, the state-of-the-art VAE model for learning disentangled representations.

For our RL experiments, we learn to solve navigation tasks using features from the trained VAE. The policies are trained in an on-policy fashion. At each time-step $t$ we receive the sensory state input $o_t$, pass it through the SRL model (i.e. a VAE) to obtain the state representation $s_t$, and feed it to a policy $\pi(s_t)$ optimized with an RL algorithm, as described in Fig.\ref{fig:srl}. For the RL algorithm, we selected the Proximal Policy Optimization (PPO) algorithm \citep{schulman2017proximal} of SRL Toolbox \citep{raffin2018s}, which in our case employs an actor-critic network that both predict the value function and a policy. We selected this method as it is a state-of-the-art policy gradient method, commonly used and robust to hyperparameter configurations. Policies are trained on-policy for $2M$ timesteps. Results are averaged over $5$ seeds, and we test the significance of our results with statistical hypothesis testing. To compare learning curves, we present smoothed (moving average) and normalized (by the maximum performance) mean reward curves.

We provide implementation details, hyperparameters configurations and neural network architectures for our experiments in Appendix B.3 (Experiment 1) and D.3 (Experiment 2).

\subsection{Evaluation}
\label{sec:lesnoms_inputs}

First, we evaluate whether the VAE successfully learned to reconstruct states, generate realistic states using sampling in the latent space and avoid catastrophic forgetting. We use visualization to measure performance. We also use the Mean Squared Error (MSE) to evaluate reconstruction quality and catastrophic forgetting over time as the environment changes.

Then, we evaluate the VAE ability to provide a state representation that enables efficient RL training, by measuring the performance of an RL agent using VAE features as input. The aim of SRL here is to have features that enable efficient behaviour learning hence the need to evaluate with RL. We use several baselines for comparison with S-TRIGGER (policy inputs are features of a VAE trained with our approach): Raw Pixels (policy inputs are raw sensory states, no information compression), Fine-tuning (policy inputs are features of a VAE sequentially fine-tuned), Source Only (policy inputs are features of a VAE trained only on first environment) and Upperbound (policy inputs are features of a VAE trained on data from all environment at once)

\section{Experiment 1: Proof of concept}

In this experiment we test S-TRIGGER on a continual learning setting with minimal changes between two environments. This experiment allows to easily visualize the failure mode of fine-tuning on this setting, and provides insights on how our method solves the presented issues.

\begin{figure}[h!]
    \centering
    \includegraphics[scale=0.3]{figures/env_exp1_reconstruction_forgetting.png}
    \caption{\textbf{Top}: The two environments considered in Experiment 1. \textbf{Bottom}: Reconstruction comparison of sensory states from environment 1 between a VAE fine-tuned on environment 2 against a VAE trained with S-TRIGGER, followed by visualization of real and generated sensory states of a VAE trained in Environment 1. Must be viewed in color.}
    \label{fig:env_explained}
\end{figure}

\subsection{Description of the environments}

We use two environments displayed in Fig.\ref{fig:env_explained}. Most elements are identical between the two worlds: both are rooms of the same size with 3 fixed obstacles, 10 randomly placed round blue obstacles and 10 randomly placed round edible items. The only variation is the color of the edibles items. The environment changes once the representation model is finished training on the first environment. The navigation task consists in collecting as many edible items as possible in $500$ timesteps. The input is the vision of the agent, i.e. a 1-D image corresponding to what the agent sees in front of it. The experimental setup is detailed in Appendix B.1.

\subsection{Results}
\label{sec:results}

\begin{figure*}[h!]
    \centering
    \includegraphics[scale=0.47]{figures/fig_exp1.pdf}
    \caption{Smoothed normalized mean reward and standard error over 5 runs of RL evaluation using PPO with different inputs. Each method is described in Sec.\ref{sec:lesnoms_inputs}.

    Dots indicates significance when testing against Fine-tuning with a Welch's
 t-test at level $\alpha = 0.05$.}
    \label{fig:rl_eval}
\end{figure*}

\subsubsection{Reconstruction evaluation and detection of environment change}

We present in Fig.\ref{fig:env_explained} a qualitative evaluation of Fine-tuning and S-TRIGGER methods on VAE. Models are sequentially trained on the first and then the second environment. Fine-tuning of VAE on the second environment leads to forgetting of the ability to reconstruct states from the first environment. On the contrary, S-TRIGGER successfully avoids this problem and the resulting VAE is able to properly reconstruct all elements of both environments, hence successful continual learning. Quantitatively (see Table \ref{tab:final_rl_perf}), the MSE of reconstruction over $500$ samples is similar for the two method on environment 2, whereas S-TRIGGER is one order of magnitude better than Fine-tuning on environment 1. This confirms our initial qualitative observations. We also present generated samples after the VAE is trained on the first environment in Fig.\ref{fig:env_explained}. The VAE is able to generate realistic sensory states that are used by S-TRIGGER to remember past environments.

Regarding environment change detection, our method tests whether a VAE trained on the first environment has a mean reconstruction error statistically different between states of environments 1 and 2. As seen in Fig.1 in Appendix,
the reconstruction error mean is significatively higher on frames of environment 2, since the VAE has only been trained on frames from environment 1. We take advantage of this observation as we use a Welch's $t$-test to compare two batches of mean VAE reconstruction error, computed on randomly collected states from each environment. The null hypothesis is rejected if the $p$-value is greater than $0.01$. We repeat this experiment $5000$ times. The test is 100\% successful when it should detect an environment change, and 99.5\% successful when it should not detect a change. Any chosen critical $p$-value between $0.05$ and $0.0001$ provide similar results. See Appendix B.2 for details.

\begin{table}[!h]
\centering

  \caption{Mean (5 seeds) final performance of RL evaluation (\textbf{Top}) and MSE reconstruction error (\textbf{Bottom}) for Experiment 1. Each method is described in Sec.\ref{sec:lesnoms_inputs}. After training, each model is used to solve a task in each environment. $^{(1)}$ indicates significance when testing against Fine-tuning.

  }
  \label{tab:final_rl_perf}

  \resizebox{\columnwidth}{!}{\begin{tabular}{ccc}
    \hline
    \multicolumn{3}{c}{RL evaluation}\\
    \hline
    Inputs for RL& Task 1& Task 2\\
    \hline
    Raw pixels & $92.30 \pm 5.8$   &  $123.95 \pm 25.6$ \\
    VAE - Source Only  & \bm{$121.25$}$ \pm 5.3$   &  $111.75 \pm 11.9$ \\
    VAE - Fine-tuning  & $96.55 \pm 5.1$   &  $172.5 \pm 11.5$ \\
    VAE - S-TRIGGER (Ours)  &  $112.85 \pm 13.2$ $^{(1)}$  & \bm{$256.95$}$ \pm 10.3$ $^{(1)}$ \\
    \\
    \hline
    \multicolumn{3}{c}{Mean Squared Error (MSE) of reconstruction}\\
    \hline
    Strategy& MSE (environment 1)& MSE (environment 2)\\
    \hline
    Fine-tuning  & $1.3 \cdot 10^{-3}$   &  $9.3 \cdot 10^{-4}$ \\
    S-TRIGGER (Ours) & $3.3 \cdot 10^{-4}$ & $6.4 \cdot 10^{-4}$ \\
    \hline
\end{tabular}}
\end{table}

\subsubsection{RL evaluation}

Learning curves and final performances are presented in Fig.\ref{fig:rl_eval} and Table \ref{tab:final_rl_perf}. We first verify that using State Representation instead of directly using the raw states is preferable in terms of final performance and sample efficiency.
We performed an additional experiment on a different color transition which verified that the results persisted, see Appendix C.

We observe that VAE's features supports a form of zero-shot transfer. Indeed, using features of a VAE trained on the first environment only (see Source Only curve in Fig.\ref{fig:rl_eval}) to learn to solve task $2$ is remarkably efficient.
Incidentally, using features of a VAE fine-tuned on the second environment to learn to solve task $1$ is also quite satisfactory, even if we previously showed that the model has catastrophically forgotten how to reconstruct states of the first environment. Hence, Fine-tuning is a much more competitive baseline compared to how naive the method is. Yet, forgetting is still apparent: there is still significant gap in performance compared to using a model trained on the first environment.

Our method S-TRIGGER partially completes this gap as it does not forget how to encode states from the first environment. Additionally, we observe on task $2$ that it largely outperforms all other methods. We infer that learning on both environments produces representations that are more suited for policy learning, a form of forward transfer.

\section{Experiment 2: Robustness tests}

In this experiment, we test the robustness of S-TRIGGER on a scenario with sequences of randomly generated mazes. The setting is more challenging compared to Experiment 1 because of both the extended number of environments (3 sequential environments) and the heavier changes between them (changes of color, location, size of all elements).

\subsection{Description of the environments}

We use sequences of three randomly generated mazes. The mazes are composed of rooms and corridors, and randomly placed edibles: fruits ($+10$ reward) and poisons ($-10$ reward). Fruits share the same color, different than poisons. In the sequence, rooms and corridors change location, size and color. The size of edibles gradually increase in the sequence, and their color changes as well. The environment changes once the representation model is finished training. In each of the three environments, the RL task is to learn how to collect fruits while avoiding poisons. The setting is illustrated in Fig.\ref{fig:env_explained_exp2}, and detailed in Appendix D.1.

\subsection{Results}
\label{sec:results2}
\begin{figure*}[!t]
    \centering
    \includegraphics[scale=0.43]{figures/fig_exp2.pdf}
    \caption{Randomly generated mazes: smoothed normalized mean reward and standard error over 5 runs of RL evaluation using PPO with VAE features as inputs. Each method is described in Sec.\ref{sec:lesnoms_inputs}.

    Dots indicates significance when testing against Fine-tuning with a Welch's t-test at level $\alpha = 0.05$.}
    \label{fig:rl_eval2}
\end{figure*}

We first evaluate the reconstruction error and the environment change detection on a set of $100$ sequences of 3 randomly generated mazes. Then, we present the Reinforcement Learning evaluation on one sequence of 3 mazes.

\subsubsection{Reconstruction evaluation and detection of environment change}

\begin{figure}[htb!]
    \centering
    \includegraphics[scale=0.27]{figures/exp2.png}
    \caption{\textbf{Top}: One of the sequences of 3 randomly generated mazes considered in Experiment 2. \textbf{Bottom}: Reconstruction of sensory states by a VAE sequentially trained with S-TRIGGER on the 3 mazes presented at the top, followed by visualization of real and generated sensory states of a VAE trained in Maze 1. Best viewed in color.}
    \label{fig:env_explained_exp2}
\end{figure}

In this experiment, we compare S-TRIGGER and Fine-tuning by sequentially training a VAE on sequences of 3 randomly generated mazes, using randomly collected sensory states. We repeat the process $100$ times with different mazes each time and present average results in Tab. \ref{tab:100_exp}. Contrary to Fine-tuning, S-TRIGGER does not forget how to reconstruct sensory states, and performs one order of magnitude better on previously encountered environments, i.e. Mazes 1 and 2. Qualitatively, we can observe how S-TRIGGER avoids catastrophic forgetting in Fig.\ref{fig:env_explained_exp2}: after encountering Maze 3, the VAE reconstruction is satisfactory on all 3 mazes, hence the state's information has been successfully compressed and this information has been kept over time. Also, the generation quality is still satisfactory, as seen in Fig.\ref{fig:env_explained_exp2}

\begin{table}[!h]
\centering

  \caption{Averaged results over $100$ independent experiments: Fine-tuning vs. S-TRIGGER trained on fixed sequences of 3 randomly generated mazes. We compare reconstruction error between the two methods on 25k sensory states (lower is better) (\textbf{Top}), and report precision and recall for the environment change detection method (\textbf{Bottom}).

  }
  \label{tab:100_exp}

  \resizebox{\columnwidth}{!}{\begin{tabular}{ccccc}
    \hline
    \multicolumn{5}{c}{Reconstruction comparison (MSE)}\\
    \hline
    Strategy& &Maze 1& Maze 2& Maze 3\\
    \hline
    Fine-tuning&  & $2.39 \cdot 10^{-2}$ &$2.41 \cdot 10^{-2}$ & $\bm{6.02 \cdot 10^{-4}}$  \\
    S-TRIGGER (Ours)& & $\bm{5.82 \cdot 10^{-3}}$ &$\bm{5.24 \cdot 10^{-3}}$ & $1.01 \cdot 10^{-3}$  \\
    \\
    \hline
    \multicolumn{5}{c}{Environment change detection (Precision, Recall, average over 4000 transitions)}\\
    \hline
    Method & \multicolumn{2}{c}{Transition 1-2} & \multicolumn{2}{c}{Transition 2-3} \\
    S-TRIGGER & \multicolumn{2}{c}{(99.82\%, 99.38\%)} & \multicolumn{2}{c}{(99.69\%, 99.29\%)} \\
    \hline
\end{tabular}}
\end{table}

Notice that the reconstruction is as satisfactory on Maze 1 than Maze 2, even though Maze 1 has been experienced by the agent more learning steps in the past than Maze 2. The VAE is still able to generate enough samples of Maze 1, and with a sufficient generation quality, to learn a satisfactory representation when Maze 3 is encountered. Also, since we obtained successful results on $100$ different sequences of 3 mazes, we can infer that S-TRIGGER is successful on any sequence of 3 randomly generated mazes. Our method is thus robust to the continual learning scenario considered.

As for the environment change detection method of S-TRIGGER, we test it on the 100 sequences, by artificially creating 400 transitions between environments per sequence (200 with a change, 200 without a change), which sums up to 4000 different transitions, see Appendix D.2 for details. We show in Tab. \ref{tab:100_exp} that the method performs quasi-perfectly well, with at least $99\%$ precision and recall. Considering the fact that it can be ran continuously for a low computational cost, the method is well suited for the considered setting.

\subsubsection{RL evaluation}

For experiments in this section, we randomly selected one sequence on 3 randomly generated mazes, and kept it fixed for all experiments. VAE and policies are all trained on this same sequence of mazes, presented in Fig.\ref{fig:env_explained_exp2}. We compare performance of policies trained on-policy with the RL algorithm PPO. Policies are trained using VAE features as inputs. VAEs are trained with three different strategies: Fine-tuning, S-TRIGGER (both sequentially trained) and Upperbound (jointly trained on data from the three mazes, not sequential). Learning curves and final performances are respectively presented in Tab. \ref{tab:final_rl_perf_exp_2} and Fig.\ref{fig:rl_eval2}. We provide an illustrative video\footnote{\url{https://youtu.be/u3d00AydDH4}} with trained policies and interpretations.

S-TRIGGER performs at least twice better than Fine-tuning on Maze 1 and 2, and performs optimally w.r.t Upperbound on Maze 1 and 3. Indeed, Upperbound has access to all data at once, which makes it an optimal solution w.r.t to using VAE features for policy learning in the considered Continual learning scenario. S-TRIGGER achieves Continual SRL: the learned representation model allows fast and efficient policy learning with a standard RL algorithm without forgetting of previously seen environments.

Compared to Experiment 1, Fine-tuning performs worse on previously seen environments, due to the increased difference between encountered environments. As seen in the supplementary video, since the size of edibles increase for each new maze, a VAE sequentially trained with Fine-tuning until Maze 3 is not trained to see small objects in Maze 1, which makes the agent almost entirely \textit{blind} in Maze 1. On the contrary, S-TRIGGER allows the VAE to see objects in all mazes, which contributes to having faster and better-performing learned policies. Fine-tuning has one of the expected shortcomings of approaches like DARLA \cite{higgins2017darla}. When there is too much change between environments, learning a sufficiently general representation that can transfer to all environment is not feasible. The representation model needs to be updated when a new environment is encountered.

\begin{table}[!h]
\centering

  \caption{Mean (5 seeds) final performance of RL evaluation for experiment 2. Each method is described in Sec.\ref{sec:lesnoms_inputs}. After training, each model is used to solve a task in each environment. $^{(1)}$ indicates significance when testing against Fine-tuning with a Welch's t-test at level $\alpha = 0.05$.\\}
  \label{tab:final_rl_perf_exp_2}

  \resizebox{\columnwidth}{!}{\begin{tabular}{cccc}
    \hline
    Inputs for RL& Maze 1& Maze 2& Maze 3\\
    \hline
    VAE - Fine-tuning  & $74.1 \pm 11.0$   &  $80.05 \pm 9.9$  & $251.25 \pm 23.4$\\
    VAE - S-TRIGGER (Ours) &  \bm{$140.3 \pm 9.4$} $^{(1)}$  & \bm{$150.45$}$ \pm 16.3$ $^{(1)}$ & $275.65 \pm 18.8$ \\
    \hline
    VAE - Upperbound & $161.35 \pm 10.6$    &  $249.95 \pm 11.1$ & $262.55 \pm 13.3$ \\
    \hline
\end{tabular}}
\end{table}

\section{Conclusion and future work}

We described S-TRIGGER, a Continual State Representation Learning method capable of learning as the environment changes discretely.
Our method automatically detects changes and relies on using generated samples of previous environments, with a fixed-size system.
It learns a unique representation model that compresses information of each encountered environments, which can be used to train policies with RL, efficiently and with high-performance.

One direction of future work is to extend S-TRIGGER to non-discrete environment changes. One way to do it would be to discretize continuous changes of the environment, but it would be preferable to extend the method so that is could smoothly handle continuous changes of environments.

\bibliographystyle{icml2019}

redpajama_set_name: RedPajamaArXiv


2003-12-19 Assigned to SCIMED LIFE SYSTEMS, INC. reassignment SCIMED LIFE SYSTEMS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHOUINARD, PAUL F., RICHARDSON, LEONARD B., HILL, JASON P., SHOEMAKER, SUSAN M.
2005-12-16 Assigned to BOSTON SCIENTIFIC SCIMED, INC. reassignment BOSTON SCIENTIFIC SCIMED, INC. MERGER, CHANGE OF NAME Assignors: SCIMED LIFE SYTEMS, INC.
A venous valve with a tubular frame that includes an outer surface and an inner surface opposite the outer surface and defining a lumen, and a cover over at least the outer surface of the tubular frame, where the cover includes surfaces defining a reversibly sealable opening for unidirectional flow of a liquid through the lumen. A system with the venous valve and a catheter including a proximal end and a distal end, the venous valve located between the proximal end and distal end of the catheter. A method including forming the venous valve and reversibly joining the venous valve and a catheter. A method including positioning at least part of the catheter including the venous valve at a predetermined location and deploying the venous valve from the catheter at the predetermined location.
The present invention relates generally to apparatus, systems, and methods for use in a lumen; and more particularly to venous valve apparatus, systems, and methods for use in the vasculature system.
The venous system of the legs uses pumps and valves to return blood to the heart. Venous valves create one way flow to prevent blood from flowing away from the heart. When valves fail, blood can pool in the lower legs resulting in swelling and ulcers of the leg. The absence of functioning venous valves can lead to chronic venous insufficiency.
Techniques for both repairing and replacing the valves exist, but are tedious and require invasive surgical procedures. Direct and indirect valvuoplasty procedures are used to repair damaged valves. Transposition and transplantation are used to replace an incompetent valve. Transposition involves moving a vein with an incompetent valve to a site with a competent valve. Transplantation replaces an incompetent valve with a harvested valve from another venous site. Prosthetic valves can be transplanted into the venous system, but current devices are not successful enough to see widespread usage.
FIGS. 1A and 1B illustrate an embodiment of a valve.
FIGS. 2A-2D illustrate an embodiment of a valve in perspective view (FIGS. 2A and 2B) and sectional view (FIGS. 2C and 2D) of the valve.
FIGS. 3A and 3B illustrate an embodiment of a valve in a compressed state (FIG. 3A) and in an expanded state (FIG. 3B).
FIGS. 4A-4D illustrate an embodiment of forming a valve.
FIGS. 5A-5C illustrate an embodiment of forming a cover.
FIGS. 6A-4E illustrate an embodiment of forming a valve.
FIGS. 7A-7D illustrate an embodiment of forming a valve.
FIG. 8 illustrates an embodiment of a system that includes a valve.
FIG. 9 illustrates an embodiment of a system that includes a valve.
FIG. 10 illustrates an embodiment of a system that includes a valve.
Embodiments of the present invention are directed to an apparatus, system, and method for valve replacement. For example, the apparatus can include a valve that can be used to replace an incompetent valve in a body lumen. Embodiments of the valve can include a tubular frame and cover that can be implanted through minimally-invasive techniques into the body lumen. In one example, embodiments of the apparatus, system, and method for valve replacement may help to maintain antegrade blood flow, while decreasing retrograde blood flow in a venous system of individuals having venous insufficiency, such as venous insufficiency in the legs.
The Figures herein follow a numbering convention in which the first digit or digits correspond to the drawing Figure number and the remaining digits identify an element or component in the drawing. Similar elements or components between different Figures may be identified by the use of similar digits. For example, 110 may reference element "10" in FIG. 1, and a similar element may be referenced as 210 in FIG. 2. As will be appreciated, elements shown in the various embodiments herein can be added, exchanged, and/or eliminated so as to provide any number of additional embodiments of valve.
FIGS. 1A-3 provide illustrations of various embodiments of a valve of the present invention. Generally, valve can be implanted within the fluid passageway of a body lumen, such as for replacement of a valve structure within the body lumen (e.g., a venous valve), to regulate the flow of a bodily fluid through the body lumen in a single direction. Figures 1A and 1B illustrate one embodiment of a venous valve 100. Venous valve 100 includes a tubular frame 102 and a cover 104 for the venous valve 100, where both the tubular frame 102 and the cover 104 can resiliently radially collapse and expand. The tubular frame 102 includes an outer surface 106 and an inner surface 108 opposite the outer surface 106. The inner surface 108 defines a lumen 110 of the venous valve 100 for passing fluid (e.g., blood) therethrough. The tubular frame 102 also includes a first end 112 and a second end 114.
In one embodiment, the cover 104 can be located over at least the outer surface 106 of the tubular frame 102. For example, the cover 104 can extend around a perimeter of the tubular frame 102 so as to completely cover the outer surface of the tubular frame 102. In other words, the cover 104 extends over the outer surface of the tubular frame 102 so that there are no exposed portions of the outer surface of the tubular frame 102. In an additional embodiment, the cover 104 can also be located over at least the inner surface 108 of the tubular frame 102. A further embodiment includes the cover 104 located over at least the outer surface 106 and the inner surface 108. The cover 104 can further include surfaces defining a reversibly sealable opening 116 for unidirectional flow of a liquid through the lumen 110. For example, the surfaces of the cover 104 can be deflectable between a closed configuration in which fluid flow through the lumen 110 can be restricted and an open configuration in which fluid flow through the lumen 110 can be permitted.
The tubular frame 102 can be formed from a wide variety of materials and in a wide variety of configurations. Generally, tubular frame 102 can have a unitary structure with an open frame configuration. For example, the open frame configuration can include frame members 117 that define openings 119 through the tubular frame 102. The tubular frame 102 can also be self-expanding. Examples of self-expanding frames include those formed from temperature-sensitive memory alloy which changes shape at a designated temperature or temperature range. Alternatively, the self-expanding frames can include those having a spring-bias. In addition, the tubular frame 102 can have a configuration that allows the frame 102 be radially expandable through the use of a balloon catheter.
In one embodiment, the tubular frame 102 can include an open frame configuration that includes a first vertex 118 and a second vertex 120 relative the first end 112 of the tubular frame 102. Tubular frame 102 can further include a first valley 122 and a second valley 124 adjacent the first end 112 relative the first vertex 118 and the second vertex 120. As illustrated in FIGS. 1A and 1B, the first vertex 118 and the second vertex 120 can be positioned opposite each other along a common axis 126. FIGS. 1A and 1B also illustrate that the first valley 122 and the second valley 124 can be positioned opposite each other and perpendicular to axis 126. Other relative positions for the first and second vertex 118 and 120, and the first and second valley 122 and 124 are also possible.
FIGS. 2A-2D illustrate an additional embodiment of a venous valve 200. FIGS. 2A and 2B provide a perspective illustration of valve 200 in an open configuration (FIG. 2A) and a closed configuration (FIG. 2B). FIGS. 2C and 2D provide a sectional view of FIGS. 2A and 2B, respectively, to more clearly illustrate the embodiment of the venous valve 200.
Venous valve 200 includes a tubular frame 202 and a cover 204 for the venous valve 200 where both the tubular frame 202 and the cover 204 can resiliently radially collapse and expand. The tubular frame 202 includes an outer surface 206 and an inner surface 208 opposite the outer surface 206. The inner surface 208 defines a lumen 210 of the venous valve 200 for passing fluid (e.g., blood) therethrough. The tubular frame 202 also includes a first end 212 and a second end 214.
In one embodiment, the cover 204 can be located over at least the outer surface 206 of the tubular frame 202. In an additional embodiment, the cover 204 can be located over at least the inner surface 208 of the tubular frame 202. A further embodiment includes the cover 204 located over at least the outer surface 206 and the inner surface 208. The cover 204 further includes surfaces defining a reversibly sealable opening 216 for unidirectional flow of a liquid through the lumen 210. For example, the surfaces of the cover 204 can be deflectable between a closed configuration in which fluid flow through the lumen 210 can be restricted and an open configuration in which fluid flow through the lumen 210 can be permitted.
Generally, tubular frame 202 can have a unitary structure with an open frame configuration. For example, the open frame configuration can include frame members 217 that define openings 219 through the tubular frame 202. The tubular frame 202 can include an open frame configuration in which the first end 212 and the second end 214 each include a plurality of end portions 221 that lay on a common plane 223. As illustrated in FIGS. 2A-2D, the plurality of end portions 221 that lay on the common plane 223. However, the plurality of end portions 221 need not all lay on the common plane 223 and 222. It is possible that one or more of the end portions 221 lay above and/or below the common plane 223.
While the tubular frame 102 and/or 202 illustrated herein is shown having a circular configuration, other configurations are also possible. For example, the tubular frame 102 and/or 202 can also include an elliptical configuration. As such, the present invention should not be limited to the illustration of the tubular frame 102 and/or 202.
The tubular frame 202 further includes elastic regions 228. Typically, the elastic regions 228 can occur at portions of the tubular frame 202 that include curves 230 in the frame members 217. The elastic regions 228 allow the valve 200 to accommodate changes in body lumen size (e.g., diameter of the body lumen) by flexing to expand and/or contract to change the diameter of the tubular frame 202. In one embodiment, the curves 230 in the frame members 217 can act as springs to allow the valve 200 to resiliently radially collapse and expand. The valve frame 202 can also provide sufficient contact and expansion force with the surface of a body lumen wall to encourage fixation of the valve 200 and to prevent retrograde flow within the body lumen. Anchoring elements (e.g., barbs) can also be included with valve 200, as will be discussed herein.
The elastic regions 228 of the valve frame 202 allow valve 200 to elastically and repeatably travel between a collapsed state and an expanded state. FIG. 3A provides an example of the valve 300 in a collapsed state, and FIG. 3B provides an example of the valve 300 in an expanded state. As shown in FIGS. 3A and 3B, the valve 300 can travel between the collapsed and the expanded state along a radial travel path 336, where there can be a change in a cross sectional area 338 of the lumen 310. For example, the valve frame 302 can travel along the radial travel path 336 so as to change a width 340 of lumen 310. This can allow the valve 300 to react appropriately to any distension and contraction of the lumen in which the valve 300 is placed.
In addition to the curves 230, the elastic regions 228 can further include, but are not limited to, other shapes for the valve frame 202 that allow for repeatable travel between the collapsed state and the expanded state. For example, the elastic regions 228 can include integrated springs having a circular or an elliptical coil configuration. Other shapes are also possible.
Frame member 217 of the valve frame 202 can include a variety of cross-sectional shapes and dimensions. For example, cross-sectional shapes for the frame member 217 can include, but are not limited to, circular, tubular, I-shaped, T-shaped, oval, trapezoidal, and triangular. The frame member 217 can also have a single cross-sectional shape (e.g., all of valve frame 202 can have a circular cross-sectional shape). In an additional embodiment, the frame member 217 can have two or more cross-sectional shapes (e.g., a circular cross-sectional shape in the elastic region 228 and a different cross-sectional shape in other regions of valve frame 202).
Valve frame 202 can also include one or more contiguous frame member 217. For example, the frame member 217 of valve frame 202 can be a single contiguous member. The single contiguous member can be bent around an elongate tubular mandrel to form the valve frame 202. The free ends of the single contiguous member can then be welded, fused, crimped, or otherwise joined together to form the valve frame 202. In an additional embodiment, the frame member 217 of valve frame 202 can be derived (e.g., laser cut, water cut) from a single tubular segment. In an alternative embodiment, methods of joining the frame member 217 to create the elastic region 228 include, but are not limited to, welding, gluing, and fusing the frame member 217. The valve frame 202 can be heat set by a method as is typically known for the material which forms the valve frame 202.
The valve frame 202 can be formed from any number of materials. For example, the valve frame 202 can be formed from a biocompatible metal, metal alloy, polymeric material, or combination thereof. As discussed herein, the valve frame 202 can be self-expanding or balloon expandable. In addition, the valve frame 202 can be configured so as to have the ability to move radially between the collapsed state and the expanded state. To accomplish this, the material used to form the valve frame 202 should exhibit a low elastic modulus and a high yield stress for large elastic strains that can recover from elastic deformations. Examples of suitable materials include, but are not limited to, medical grade stainless steel (e.g., 316L), titanium, tantalum, platinum alloys, niobium alloys, cobalt alloys, alginate, or combinations thereof. In an additional embodiment, the valve frame 202 may be formed from a shape-memory material, such as shape memory plastics, polymers, and thermoplastic materials which are inert in the body. Shaped memory alloys having superelastic properties generally made from specific ratios of nickel and titanium, commonly known as nitinol, are also possible materials. Other materials are also possible.
Tubular frame 202 can be expanded to provide lumen 210 having any number of sizes. For example, the size of lumen 210 can be determined based upon the type of body lumen and the body lumen size in which the valve 200 is to be placed. In an additional example, there can also be a minimum value for the width 240 for the tubular frame 202 that ensures that the tubular frame 202 will have an appropriate expansion force against the inner wall of the body lumen in which the valve 200 is being placed. The tubular frame 202 can also include a longitudinal length 242.
In one embodiment, the valve frame 202 can further include one or more anchoring elements. For example, the one or more anchoring elements can include, but are not limited to, one or more barbs 243 projecting from the outer surface 206 of the tubular frame 202. The valve 200 can further include one or more radiopaque markers (e.g., tabs, sleeves, welds). For example, one or more portions of the valve frame 202 can be formed from a radiopaque material. Radiopaque markers can be attached to and/or coated onto one or more locations along the valve frame 202. Examples of radiopaque material include, but are not limited to, gold, tantalum, and platinum. The position of the one or more radiopaque markers can be selected so as to provide information on the position, location and orientation of the valve 200 during its implantation.
As discussed herein, valve 100 further includes cover 104 having surfaces defining the reversibly sealable opening 116 for unidirectional flow of a liquid through the lumen 110. For the embodiment illustrated in FIGS. 1A and 1B, the cover 104 extends over at least a portion of the tubular frame 102 to a first connection point 144 and a second connection point 146 on the tubular frame 102. In one example, the first connection point 144 and the second connection point 146 can be located at the first vertex 118 and the second vertex 120 of the tubular frame 102. The cover 104 extends between the first connection point 144 and the second connection point 146 to provide a first valve leaflet 132 and a second valve leaflet 134. The first valve leaflet 132 and the second valve leaflet 134 can form the reversibly sealable opening 116 extending between the first connection point 144 and the second connection point 146. So, for the example shown in FIG. 1 the first valve leaflet 132 and the second valve leaflet 134 form the reversibly sealable opening 116 extending between the first vertex 118 and the second vertex 120 of the tubular frame 102.
As illustrated, the first valve leaflet 132 and the second valve leaflet 134 include a region 148 of the cover 104 that can move relative the tubular frame 102. The region 148 of the cover 104 can be unbound (i.e., unsupported) by the tubular frame 102 and extends between the first connection point 144 and the second connection point 146 of the valve 100. This configuration permits the reversibly sealable opening 116 to open and close in response to the fluid pressure differential across the valve leaflets 132 and 134.
For example, under antegrade fluid flow (i.e., positive fluid pressure) from the first end 112 towards the second end 114 of the valve 100, the first and second valve leaflets 132 and 134 can expand toward the inner surface 108 to create an opening through which fluid is permitted to move. In one example, the first valve leaflet 132 and the second valve leaflet 134 can each expand to define a semi-tubular structure when fluid opens the reversibly sealable opening 116. An example of the open configuration for the valve is shown in FIGS. 1A and 2A.
Under a retrograde fluid flow (i.e., negative fluid pressure) from the second end 114 towards the first end 112, the first and second valve leaflets 132 and 134 can move away from the inner surface 108 as the valve leaflets 132 and 134 begin to close valve 100. In one example, a pocket exists between the frame 102 and each of the first and second valve leaflets 132 and 134. The pocket allows fluid from the retrograde flow to develop pressure on a first major face 150 of the first and second valve leaflets 132 and 134. As fluid pressure develops, the first and second valve leaflets 132 and 134 collapse, closing the reversibly sealable opening 116 to create a seal 154, thereby restricting retrograde fluid flow through the valve 100. In one example, the seal 154 can be created by the joining of a sealing surface 156 of the first and second valve leaflets 132 and 134. In the closed configuration, the first and second valve leaflets 132 and 134 can each have a concave structure 158 when fluid closes the reversibly sealable opening 116. An example of the closed configuration for the valve is shown in FIGS. 1B and 2B.
Valve 100 provides an embodiment in which the surfaces defining the reversibly sealable opening 116 provide a bi-leaflet configuration (i.e., a bicuspid valve) for valve 100. Although the embodiments in FIGS. 1A-B and 2A-D illustrate and describe a bi-leaflet configuration for the valve of the present invention, designs employing a different number of valve leaflets (e.g., tri-leaflet valve) are possible. For example, additional connection points (e.g., three or more) could be used to provide additional valve leaflets (e.g., a tri-leaflet valve).
The first valve leaflet 132 and the second valve leaflet 134 can have a variety of sizes and shapes. For example, each of the first valve leaflet 132 and the second valve leaflet 134 can have a similar size and shape. In addition, each of the first valve leaflet 132 and the second valve leaflet 134 can include opposed first and second major surfaces 150 and 152, respectively. Each first major surface 150 of the first valve leaflet 132 and the second valve leaflet 134 can be oriented to face the second end 114 of valve 100.
Each of the first valve leaflet 132 and the second valve leaflet 134 can further provide the sealing surface 156 formed by portions of the first valve leaflet 132 and the second valve leaflet 134, where the sealing surface 156 can engage to define the closed configuration (FIG. 1B) of valve 100. Sealing surface 156 of the first valve leaflet 132 and the second valve leaflet 134 can separate to provide for an open configuration (FIG. 1A) of valve 100. In an additional example, each of the first valve leaflet 132 and the second valve leaflet 134 need not have a similar size and shape (i.e., the valve leaflets can have a different size and shape with respect to each other).
In one embodiment, each of the first valve leaflet 132 and the second valve leaflet 134 includes sufficient excess material spanning tubular frame 102 such that fluid pressure (e.g., antegrade flow) acting on the second major surface 152 of the first valve leaflet 132 and the second valve leaflet 134 forces the valve 100 into an open configuration (FIG. 1A). The first valve leaflet 132 and the second valve leaflet 134 further include arcuate edges 160 and 162 that are positioned adjacent each other along a substantially catenary curve between the connection point 144 and the second connection point 146 in the closed configuration (FIG. 1B) of valve 100. Similarly, arcuate edges 160 and 162 can define opening 116 when the valve 100 is in the open configuration (FIG. 1A).
In an additional embodiment, in the open configuration the portion of the cover 104 forming the first valve leaflet 132 and the second valve leaflet 134 provides sufficient excess material spanning between the connection point 144 and the second connection point 146 to allow the first and second major surfaces 150 and 152 to take on a semi-tubular structure 164, as shown in FIG. 1A, when fluid pressure opens the valve 100. In an additional embodiment, arcuate edge 160 and 162 of valve 100 can open to approximately the full inner diameter of a body lumen.
Each of the second major surfaces 152 of the first valve leaflet 132 and the second valve leaflet 134 can further include a curve imparted thereto so as to provide the second major surface 152 with the concave structure 158. The concave structure 158 allows the first valve leaflet 132 and the second valve leaflet 134 to better collect retrograde fluid flow to urge the first valve leaflet 132 and the second valve leaflet 134 towards the closed configuration. For example, as retrograde flow begins, the first valve leaflet 132 and the second valve leaflet 134 respond by moving towards the center of valve 100. As the first valve leaflet 132 and the second valve leaflet 134 approach the center of the device the sealing surfaces 156 make sufficient contact to effectively close valve 100 and restrict retrograde fluid flow.
In an additional embodiment, the first valve leaflet 132 and the second valve leaflet 134 can include one or more support structures, where the support structures can be integrated into and/or onto the valve leaflets 132 and 134. For example, the first valve leaflet 132 and the second valve leaflet 134 can include one or more support ribs having a predetermined shape. In one embodiment, the predetermined shape of the support ribs can include a curved bias so as to provide the first valve leaflet 132 and the second valve leaflet 134 with a curved configuration. Support ribs can be constructed of a flexible material and have dimensions (e.g., thickness, width and length) and cross-sectional shape that allows the support ribs to be flexible when the first valve leaflet 132 and the second valve leaflet 134 are urged into an open position, and stiff when the first valve leaflet 132 and the second valve leaflet 134 are urged into a closed position upon experiencing sufficient back flow pressure from the direction downstream from the valve. In an additional embodiment, support ribs can also be attached to valve frame 102 so as to impart a spring bias to the valve leaflets in either the open or the closed configuration.
In one embodiment, the material of the first valve leaflet 132 and the second valve leaflet 134 can be sufficiently thin and pliable so as to permit radially-collapsing of the valve leaflets for delivery by catheter to a location within a body lumen. The first valve leaflet 132 and the second valve leaflet 134 can be constructed of a fluid-impermeable biocompatible material that can be either synthetic or biologic. Possible synthetic materials include, but are not limited to, expanded polytetrafluoroethylene (ePTFE), polytetrafluoroethylene (PTFE), polystyrene-polyisobutylene-polystyrene (SIBS), polyurethane, segmented poly(carbonate-urethane), Dacron, polyethlylene (PE), polyethylene terephthalate (PET), silk, urethane, Rayon, Silicone, or the like. Possible biologic materials include, but are not limited to, autologous, allogeneic or xenograft material. These include explanted veins and decellularized basement membrane materials, such as small intestine submucosa (SIS) or umbilical vein.
As discussed herein, the cover 104 can be located over at least the outer surface 106 of the tubular frame 102. FIGS. 1A and 1B illustrate this embodiment. In an additional embodiment, the cover 104 can also be located over at least the inner surface 108 of the tubular frame 102. FIGS. 2A-2D illustrate of this embodiment. In one example, the cover 104 can further be located over the openings 119 defined by the members 117 of the tubular frame 102. The cover 104 can also be joined to itself through the openings 119 so as to fully or partially encase the tubular frame 102. Numerous techniques may be employed to laminate or bond cover 104 on the outer surface 106 and/or the inner surface 108 of the tubular frame 102, including heat setting, adhesive welding, application of uniform force and other bonding techniques. Additionally, the cover 104 may be folded over the first end of the tubular frame 102 to provide the cover 104 on both the outer surface 106 and the inner surface 108. Cover 104 can also be joined to itself and/or the members 117 according to the methods described in U.S. Patent Application Publication US 2002/0178570 to Sogard et al., which is hereby incorporated by reference in its entirety.
The cover 104 can also be coupled to the connection points so as to form the valve leaflets, as discussed herein. In one embodiment, the cover 104 can be in the form of a sheet or a sleeve of material, as discussed herein, which can be connected to the tubular frame 102. Alternatively, the cover 104 can initially be in the form of a liquid that can be used to cast and/or form the cover over the tubular frame 102. Other forms, including intermediate forms, of the cover 104 are also possible.
The cover 104 can be coupled to the tubular frame 102, including the connection points 144 and 146, in a variety of ways so as to provide the various embodiments of the valve of the present invention. For example, a variety of fasteners can be used to couple the cover 104 to the tubular frame 102 so as to form the valve 100. Suitable fasteners can include, but are not limited to, biocompatible staples, glues, sutures or combinations thereof. In an additional embodiment, the cover 104 can be coupled to the tubular frame 102 through the use of heat sealing, solvent bonding, adhesive bonding, or welding cover 104 to either a portion of the cover 104 (i.e., itself) and/or the tubular frame 102.
The cover 104, including the valve leaflets 132 and 134, may also be treated and/or coated with any number of surface or material treatments. For example, the cover 104 can be treated with one or more biologically active compounds and/or materials that may promote and/or inhibit endothelization and/or smooth muscle cell growth of the cover 104, including the valve leaflets 132 and 134. Similarly, the cover 104 may be seeded and covered with cultured tissue cells (e.g., endothelial cells) derived from a either a donor or the host patient which are attached to the valve leaflets 132 and 134. The cultured tissue cells may be initially positioned to extend either partially or fully over the valve leaflets 132 and 134.
Cover 104, in addition to forming valve leaflets 132 and 134, can also be capable of inhibiting thrombus formation. Additionally, cover 104 may either prevent or facilitate tissue ingrowth therethrough, as the particular application for the valve 100 may dictate. For example, cover 104 on the outer surface 106 may be formed from a porous material to facilitate tissue ingrowth therethrough, while cover 104 on the inner surface 108 may be formed from a material or a treated material which inhibits tissue ingrowth.
The embodiments of the valve of the present invention can be formed in any number of ways. For example, FIGS. 4A-4D provide an illustration of a method for forming valve 400. FIG. 4A illustrates one embodiment of cover 404 having a tubular configuration that includes a symmetrical portion 466 and a tapering portion 468. In the present example, the cover 404 can be formed from a continuous piece of tubular material. In one example, the tapering portion 468 can be formed by removing predetermined portions of the tubular material from the continuous piece of tubular material.
For example, FIGS. 5A-5C illustrate one embodiment of creating a tapering portion 568 for the cover 504. Cover 504 is shown in a flattened configuration to better illustrate forming the tapering portion 568 according to the present embodiment. As FIG. 5A illustrates, cover 504 can have an initial symmetrical configuration. FIG. 5A also illustrates that a portion 569, defined by line 570, of the cover 504 in the symmetrical configuration for removal in forming the tapering portion. FIG. 5B shows the portion 569 removed from the cover 504. Once removed, the cover 504 includes edges 571 that are joined to form the tapering portion 568. FIG. 5C provides an example of edges 571 joined for the tapering portion 568. In one example, the edges 571 can be joined in any number of ways, including any of those recited herein.
Other ways of forming the tapering portion 468 during the manufacturing of the cover 404 also exist. For example, the cover 404 can be cast, extruded, or molded (including blow molded) to form the tapering portion 468. In addition, tapering portion 468 can be provided separately from the symmetrical portion 466. In other words, each portion (e.g., tapering portion 468 and the symmetrical portion 466) are separate pieces that can either be then be joined, by method described herein, or used separately in forming valve 400.
In an alternative embodiment, the tapering portion 468 can be created as a result of at least a portion of the cover 404 being stretched over the tubular frame 402. For example, the cover 404 can have a cylindrical shape of essentially uniform inner diameter, where the inner diameter is less than an outer diameter of the tubular frame 402. The cover 404 can be stretched over the outer surface 406 of the tubular frame 402 such that the tapering portion 468 results as the cover 404 transitions from its stretched portion over the tubular frame 402 to the unstretched portion of the cover 404.
Referring again to FIGS. 4B-4D, the symmetrical portion 466 of the cover 404 can be placed around the outer surface 406 of the tubular frame 402. For example, the cover 404 can stretched slightly to allow the tubular frame 402 to be placed within the symmetrical portion 466 of the cover 404. Alternatively, the outer diameter of the tubular frame 402 could be enlarged so as to place the symmetrical portion 466 of the cover 404 around the outer surface 406 of the tubular frame 402. Other ways of placing the symmetrical portion 466 of the cover 404 around the outer surface 406 of the tubular frame 402 are also possible.
As FIG. 4C illustrates, a free end 472 of the tapering portion 468 can be folded and drawn into the lumen 410 of the tubular frame 402. The free end 472 is advanced toward the second end 414, so that the cover 404 extends over at least a portion of the inner surface 408 of the tubular frame 402. The free end 472 of the cover 404 can extend over at least the portion of the inner surface 408 to at least a first connection point 444 and a second connection point 446 on the tubular frame 402. As FIG. 4D illustrates, the cover 404 can be attached at the first connection point 444 and the second connection point 446 to form the first valve leaflet 432 and the second valve leaflet 434. As discussed herein, the first valve leaflet 432 and the second valve leaflet 434 include sealing surfaces 456 defining the reversibly sealable opening 416 for unidirectional flow of liquid through the lumen 410.
FIG. 4D further illustrates the cover 404 located over the outer surface 406 may be affixed to the cover 404 over at least a portion of the inner surface 408 through the openings 419 and/or affixed to the members 417 of the tubular frame 402. In addition, the cover 404 located over the outer surface 406 may be affixed at least to the members 417 of the tubular frame 402.
FIG. 6A-6E provides an alternative embodiment of forming a valve 600. In the present embodiment, the cover 604 can include at least two separate portions (e.g., symmetrical portion 666 shown in FIG. 6A and tapering portion 668 shown in FIG. 6B) that are affixed to the tubular frame 602 to form the valve 600. For example, the cover 604 can be formed from a symmetrical portion 666, shown in FIG. 6A, that is separate from a tapering portion 668, shown in FIG. 6B.
The tapering portion 668 and the symmetrical portion 666 can be affixed to the tubular frame 602 and to each other, as described herein, so as to form valve 600. For example, FIGS. 6C-6D illustrate symmetrical portion 666 affixed to tubular frame 602, with tapering portion 668 being inserted into lumen 610. FIG. 6E illustrates valve 600 upon affixing the tapering portion 668 to the symmetrical portion 666, the tubular frame 602 and the first and second connection points 644 and 646.
FIGS. 7A-7D illustrate an additional embodiment of forming a valve 700 according to the present invention. The valve 700 includes tubular frame 702 (FIG. 7A) and cover 704 (FIG. 7B). As discussed herein, tubular frame 702 has an open frame configuration that includes the first vertex 718 and a second vertex 720 relative the first end 712 of the tubular frame 702. In the present example, the cover 704 can have a tubular configuration, where the cover 704 can be positioned around and affixed to the tubular frame 702 (FIGS. 7C-7D) so as to form valve 700. For example, cover 704 and/or tubular frame 702 can be stretched so as to affix the cover 704 and the tubular frame 702. In addition, the cover 704 can be affixed to the tubular frame 702 by additional mechanical and/or chemical approaches, as discussed herein.
FIG. 8 illustrates one embodiment of a system 882. System 882 includes valve 800, as described herein, reversibly joined to catheter 874. The catheter 874 includes an elongate body 876 having a proximal end 878 and a distal end 879, where valve 800 can be located between the proximal end 878 and distal end 879. The catheter 874 can further include a lumen 880 longitudinally extending to the distal end 879. In one embodiment, lumen 880 extends between proximal end 878 and distal end 879 of catheter 874. The catheter 874 can further include a guidewire lumen 881 that extends within the elongate body 876, where the guidewire lumen 881 can receive a guidewire for positioning the catheter 874 and the valve 800 within a body lumen (e.g., a vein of a patient).
The system 882 can further include a deployment shaft 883 positioned within lumen 880, and a sheath 884 positioned adjacent the distal end 879. In one embodiment, the valve 800 can be positioned at least partially within the sheath 884 and adjacent the deployment shaft 883. The deployment shaft 883 can be moved within the lumen 880 to deploy valve 800. For example, deployment shaft 883 can be used to push valve 800 from sheath 884 in deploying valve 800.
FIG. 9 illustrates an additional embodiment of the system 982. The catheter 974 includes elongate body 976, lumen 980, a retraction system 986 and a retractable sheath 987. The retractable sheath 987 can be positioned over at least a portion of the elongate body 976, where the retractable sheath 987 can move longitudinally along the elongate body 976. The valve 900 can be positioned at least partially within the retractable sheath 987, where the retractable sheath 987 moves along the elongate body 976 to deploy the valve 900. In one embodiment, retraction system 986 includes one or more wires coupled to the retractable sheath 987, where the wires are positioned at least partially within and extend through lumen 980 in the elongate body 976. Wires of the retraction system 986 can then be used to retract the retractable sheath 987 in deploying valve 900.
FIG. 10 illustrates an additional embodiment of the system 1082. The catheter 1074 includes elongate body 1076, an inflatable balloon 1088 positioned adjacent the distal end 1079, and a lumen 1080 longitudinally extending in the elongate body 1076 of the catheter 1074 from the inflatable balloon 1088 to the proximal end 1078. In the present example, the inflatable balloon 1088 can be at least partially positioned within the lumen 1010 of the valve 1000. The inflatable balloon 1088 can be inflated through the lumen 1080 to deploy the valve 1000.
The embodiments of the present invention further include methods for forming the valve of the present invention, as discussed herein. For example, the valve can be formed from the tubular frame and the cover over at least the outer surface of the tubular frame, where the cover includes surfaces defining the reversibly sealable opening for unidirectional flow of a liquid through the lumen. In an additional example, the valve can be reversibly joined to the catheter, which can include a process of altering the shape of the valve from a first shape, for example an expanded state, to the compressed state, as described herein.
For example, the valve can be reversibly joined with the catheter by positioning valve in the compressed state at least partially within the sheath of the catheter. In one embodiment, positioning the valve at least partially within the sheath of the catheter includes positioning the valve in the compressed state adjacent the deployment shaft of the catheter. In an another embodiment, the sheath of the catheter functions as a retractable sheath, where the valve in the compressed state can be reversibly joined with the catheter by positioning the valve at least partially within the reversible sheath of the catheter. In a further embodiment, the catheter can include an inflatable balloon, where the balloon can be positioned at least partially within the lumen of the valve, for example, in its compressed state.
The embodiments of the valve described herein may be used to replace, supplement, or augment valve structures within one or more lumens of the body. For example, embodiments of the present invention may be used to replace an incompetent venous valve and help to decrease backflow of blood in the venous system of the legs.
In one embodiment, the method of replacing, supplementing, and/or augmenting a valve structure can include positioning at least part of the catheter including the valve at a predetermined location within the lumen of a body. For example, the predetermined location can include a position within a body lumen of a venous system of a patient, such as a vein of a leg.
In one embodiment, positioning the catheter that includes the valve within the body lumen of a venous system includes introducing the catheter into the venous system of the patient using minimally invasive percutaneous, transluminal catheter based delivery system, as is known in the art. For example, a guidewire can be positioned within a body lumen of a patient that includes the predetermined location. The catheter, including valve, as described herein, can be positioned over the guidewire and the catheter advanced so as to position the valve at or adjacent the predetermined location. In one embodiment, radiopaque markers on the catheter and/or the valve, as described herein, can be used to help locate and position the valve.
The valve can be deployed from the catheter at the predetermined location in any number of ways, as described herein. In one embodiment, valve of the present invention can be deployed and placed in any number of vascular locations. For example, valve can be deployed and placed within a major vein of a patient's leg. In one embodiment, major veins include, but are not limited to, those of the peripheral venous system. Examples of veins in the peripheral venous system include, but are not limited to, the superficial veins such as the short saphenous vein and the greater saphenous vein, and the veins of the deep venous system, such as the popliteal vein and the femoral vein.
As discussed herein, the valve can be deployed from the catheter in any number of ways. For example, the catheter can include the retractable sheath in which valve can be at least partially housed, as discussed herein. Valve can be deployed by retracting the retractable sheath of the catheter, where the valve self-expands to be positioned at the predetermined location. In an additional example, the catheter can include a deployment shaft and sheath in which valve can be at least partially housed adjacent the deployment shaft, as discussed herein. Valve can be deployed by moving the deployment shaft through the catheter to deploy valve from the sheath, where the valve self-expands to be positioned at the predetermined location. In an additional embodiment, the valve can be deployed through the use of an inflatable balloon.
Once implanted, the valve can provide sufficient contact and expansion force against the body lumen wall to prevent retrograde flow between the valve and the body lumen wall. For example, the valve can be selected to have a larger expansion diameter than the diameter of the inner wall of the body lumen. This can then allow valve to exert a force on the body lumen wall and accommodate changes in the body lumen diameter, while maintaining the proper placement of valve. As described herein, the valve can engage the lumen so as to reduce the volume of retrograde flow through and around valve. It is, however, understood that some leaking or fluid flow may occur between the valve and the body lumen and/or through valve leaflets.
While the present invention has been shown and described in detail above, it will be clear to the person skilled in the art that changes and modifications may be made without departing from the spirit and scope of the invention. As such, that which is set forth in the foregoing description and accompanying drawings is offered by way of illustration only and not as a limitation. The actual scope of the invention is intended to be defined by the following claims, along with the full range of equivalents to which such claims are entitled.
In addition, one of ordinary skill in the art will appreciate upon reading and understanding this disclosure that other variations for the invention described herein can be included within the scope of the present invention. For example, the tubular frame 102 and/or the cover 104 can be coated with a non-thrombogenic biocompatible material, as are known or will be known.
a cover over the tubular frame, wherein the cover includes surfaces defining a reversibly sealable opening for unidirectional flow of a liquid through the lumen.
2. The venous valve of claim 1, wherein the cover over the tubular frame includes the cover over the outer surface of the tubular frame.
3. The venous valve of claim 1, wherein the tubular frame includes a first vertex and a second vertex relative a first end of the tubular frame, wherein the first vertex and the second vertex are positioned opposite each other along a common axis.
4. The venous valve of claim 3, wherein the cover extends to at least the first vertex and the second vertex to form a first valve leaflet and a second valve leaflet between the first vertex and the second vertex.
5. The venous valve of claim 4, wherein the first valve leaflet and the second valve leaflet form the reversibly sealable opening extending between the first vertex and the second vertex.
6. The venous valve of claim 1, further including the cover over the inner surface of the tubular frame, wherein the cover over the inner surface extends to at least a first connection point and a second connection point on the tubular frame to form a first valve leaflet and a second valve leaflet.
7. The venous valve of claim 6, wherein the first valve leaflet and the second valve leaflet form the reversibly sealable opening extending between the first connection point and the second connection point.
a cover extending over at least the outer surface of the tubular frame to the first vertex and the second vertex to form a first valve leaflet and a second valve leaflet between the first vertex and the second vertex, wherein the first valve leaflet and the second valve leaflet includes a surface defining a reversibly sealable opening for unidirectional flow of a liquid through the lumen.
9. The venous valve of claim 8, wherein the first valve leaflet and the second valve leaflet form the reversibly sealable opening extending between the first vertex and the second vertex.
10. The venous valve of claim 8, wherein the tubular frame moves between a collapsed state and an expanded state.
11. The venous valve of claim 8, wherein the first valve leaflet and the second valve leaflet include support ribs having a predefined shape.
a cover over at least the outer surface and the inner surface of the tubular frame, wherein the cover over the inner surface extends to at least a first connection point and a second connection point on the tubular frame to form a first valve leaflet and a second valve leaflet, the first valve leaflet and the second valve leaflet include surfaces defining a reversibly sealable opening for unidirectional flow of a liquid through the lumen.
13. The venous valve of claim 12, wherein the first valve leaflet and the second valve leaflet form the reversibly sealable opening extending between the first connection point and the second connection point.
14. The venous valve of claim 12, wherein the tubular frame moves between a collapsed state and an expanded state.
15. The venous valve of claim 12, wherein the first valve leaflet and the second valve leaflet include support ribs having a predefined shape.
a catheter including a proximal end and a distal end, the venous valve located between the proximal end and distal end of the catheter.
17. The system of claim 16, wherein the catheter includes an elongate body having a lumen longitudinally extending to the distal end, a deployment shaft positioned within the lumen, and a sheath positioned adjacent the distal end, the venous valve positioned at least partially within the sheath and adjacent the deployment shaft, wherein the deployment shaft moves within the lumen to deploy the venous valve.
18. The system of claim 16, wherein the catheter includes an elongate body and a retractable sheath over at least a portion of the elongate body, the venous valve positioned at least partially within the retractable sheath, wherein the retractable sheath moves along the elongate body to deploy the venous valve.
19. The system of claim 16, wherein the catheter includes an inflatable balloon positioned adjacent the distal end and a lumen longitudinally extending in an elongate body of the catheter from the inflatable balloon to the distal end, the inflatable balloon at least partially positioned within the lumen of the venous valve, where the inflatable balloon inflates to deploy the venous valve.
20. The system of claim 16, wherein the tubular frame includes a first vertex and a second vertex relative a first end of the tubular frame, wherein the first vertex and the second vertex are positioned opposite each other along a common axis, where the cover extends to at least the first vertex and the second vertex to form a first valve leaflet and a second valve leaflet between the first vertex and the second vertex, wherein the first valve leaflet and the second valve leaflet includes a surface defining a reversibly sealable opening for unidirectional flow of a liquid through the lumen.
21. The system of claim 16, wherein the cover extends over at least a portion of the inner surface of the tubular frame, wherein the cover over the at least the portion of the inner surface extends to at least a first connection point and a second connection point on the tubular frame to form a first valve leaflet and a second valve leaflet, wherein the first valve leaflet and the second valve leaflet includes a surface defining a reversibly sealable opening for unidirectional flow of a liquid through the lumen.
reversibly joining the venous valve and a catheter.
23. The method of claim 22, wherein reversibly joining the venous valve and the catheter includes positioning the venous valve at least partially within a sheath of the catheter.
24. The method of claim 23, wherein positioning the venous valve at least partially within a sheath of the catheter include positioning the venous valve adjacent a deployment shaft of the catheter.
25. The method of claim 23, wherein the sheath includes a retractable sheath of the catheter.
inflating the balloon to deploy the vascular valve.
27. The method of claim 22, wherein the tubular frame includes a first vertex and a second vertex relative a first end of the tubular frame, wherein the first vertex and the second vertex are positioned opposite each other along a common axis, where the cover extends to at least the first vertex and the second vertex to form a first valve leaflet and a second valve leaflet between the first vertex and the second vertex, wherein the first valve leaflet and the second valve leaflet includes a surface defining a reversibly sealable opening for unidirectional flow of a liquid through the lumen.
28. The method of claim 22, wherein the cover extends over at least a portion of the inner surface of the tubular frame, wherein the cover over the at least the portion of the inner surface extends to at least a first connection point and a second connection point on the tubular frame to form a first valve leaflet and a second valve leaflet, wherein the first valve leaflet and the second valve leaflet includes a surface defining a reversibly sealable opening for unidirectional flow of a liquid through the lumen.
deploying the venous valve from the catheter at the predetermined location.
30. The method of claim 29, wherein positioning at least part of the catheter at the predetermined location includes positioning at least part of the catheter within a vein of a leg.
31. The method of claim 29, wherein the catheter includes a retractable sheath, the venous valve at least partially housed within the retractable sheath, and deploying the venous valve from the catheter at the predetermined location includes retracting the retractable sheath of the catheter.
32. The method of claim 29, wherein the catheter includes a deployment shaft and a sheath, the venous valve at least partially housed within the sheath adjacent the deployment shaft, and deploying the venous valve from the catheter at the predetermined location includes moving the deployment shaft to deploy the venous valve from the sheath of the catheter.
33. The method of claim 29, wherein the tubular frame includes a first vertex and a second vertex relative a first end of the tubular frame, wherein the first vertex and the second vertex are positioned opposite each other along a common axis, where the cover extends to at least the first vertex and the second vertex to form a first valve leaflet and a second valve leaflet between the first vertex and the second vertex, wherein the first valve leaflet and the second valve leaflet includes a surface defining a reversibly sealable opening for unidirectional flow of a liquid through the lumen.
34. The method of claim 29, wherein the cover extends over at least a portion of the inner surface of the tubular frame, wherein the cover over the at least the portion of the inner surface extends to at least a first connection point and a second connection point on the tubular frame to form a first valve leaflet and a second valve leaflet, wherein the first valve leaflet and the second valve leaflet includes a surface defining a reversibly sealable opening for unidirectional flow of a liquid through the lumen.
deploying the venous valve includes inflating the balloon to deploy the vascular valve.
ITBO20030631A1 (en) 2003-10-23 2005-04-24 Roberto Erminio Parravicini valvular prosthetic apparatus, in particular for cardiac applications.


Two words: Winona Ryder. Perhaps you remember the girl from films such as Reality Bites or Edward Scissorhands. Or, maybe you recall her stint as a badass shoplifter in 2001. It doesn't matter. The actress, who appeared in Black Swan and now The Dilemma, is apparently attempting to make a comeback, and we can't help but raise our glasses, hide our valuables and toast the woman for her moxie.
So it's no surprise that the latest comedy from Vince Vaughn and Kevin James is completely tolerable. Your kids will laugh, your wife will cry and you'll be singing My Sharona all the way home. All in all, it's a win-win situation.
The Dilemma, of course, doesn't attempt to break any boundaries – Vaughn plays Ronny, a lifelong bachelor whose business partner, Nick (James), appears to be married to the perfect woman, Geneva (Ryder). However, when Ronny catches Nick's wife smooching a younger man, he is conflicted about what to do.
Yep, that's it. That's is the entire story. And yet, somehow it comes together. Of course, both Vaughn and James have seen brighter days – Swingers and King of Queens are both dude-friendly gems that never fail to entertain – but the pairing of the two allows everything to click.
Another perk – Jennifer Connelly, who plays Ronny's girlfriend, Beth. While most dads would rather gouge their eyes out than be dragged to films such as A Beautiful Mind and He's Just Not That Into You, the casting of the raven-haired hottie in The Dilemma had us ordering tickets early, just to make sure we get a good seat. Consider her the "bad husband whisperer."
Child-friendly? The Dilemma tells the story of two buddies who own a car design business together. When Ronny, a cad, discovers that Nick's wife is hooking up with another dude, he runs amok trying to decide how to handle it. The film contains adult language and sexual content. Recommended for ages 13 and up.