Vision-Language Navigation with Continual Learning
Zhiyuan Li Yanfeng Lv Ziqin Tu Di Shang Hong Qiao


BMI Prediction from  Handwritten English Characters Using a Convolutional Neural Network





    N. T. Diba1, N. Akter2, S. A. H. Chowdhury3
Dept. of Electronics & Telecommunication Engineering 

Rajshahi University of Engineering & Technology

Rajshahi, Bangladesh

nishattasnimediba09@gmail.com1,
 nasrinmanha123@gmail.com2, arif.1968.ruet@gmail.com3
J. E. Giti4
Dept. of Electrical & Electronic Engineering 

Rajshahi University of Engineering & Technology

Rajshahi, Bangladesh

jishan.e.giti@gmail.com4

    September 9, 2024
===============================================================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 








Vision-language navigation (VLN) is a critical domain within embedded intelligence, requiring agents to navigate 3D environments based on natural language instructions. Traditional VLN research has focused on improving environmental understanding and decision accuracy. However, these approaches often exhibit a significant performance gap when agents are deployed in novel environments, mainly due to the limited diversity of training data. Expanding datasets to cover a broader range of environments is impractical and costly.






We propose the Vision-Language Navigation with Continual Learning (VLNCL) paradigm to address this challenge. In this paradigm, agents incrementally learn new environments while retaining previously acquired knowledge. VLNCL enables agents to maintain an environmental memory and extract relevant knowledge, allowing rapid adaptation to new environments while preserving existing information.

We introduce a novel dual-loop scenario replay method (Dual-SR) inspired by brain memory replay mechanisms integrated with VLN agents. This method facilitates consolidating past experiences and enhances generalization across new tasks. By utilizing a multi-scenario memory buffer, the agent efficiently organizes and replays task memories, thereby bolstering its ability to adapt quickly to new environments and mitigating catastrophic forgetting.


Our work pioneers continual learning in VLN agents, introducing a novel experimental setup and evaluation metrics. We demonstrate the effectiveness of our approach through extensive evaluations and establish a benchmark for the VLNCL paradigm. Comparative experiments with existing continual learning and VLN methods show significant improvements, achieving state-of-the-art performance in continual learning ability and highlighting the potential of our approach in enabling rapid adaptation while preserving prior knowledge.




















§ INTRODUCTION


Vision-Language Navigation (VLN) <cit.> is crucial for the embedding intelligence field. The agent follows natural language instructions and moves around in the 3D environment. By integrating natural language processing, visual perception, and decision-making, the agent could navigate to the destination. Most research in VLN focuses on environment understanding ability improvement <cit.> and accuracy of target decision policy <cit.>. 

While these advancements have significantly improved VLN performance, there is still a critical issue: the generalization of agents to diverse unseen scenes, which is essential for real-world applications.
In practical scenarios, agents must continually adapt to new environments while retaining the knowledge acquired from previous tasks.



The significant performance gap between seen and unseen <cit.> environments underscores this challenge. The primary cause of this issue is the limited availability of diverse environmental data, which constrains the agents' ability to generalize effectively <cit.>.

Yet, massively expanding the dataset with various environments is unrealistic and expensive <cit.>. Therefore, we consider an alternative approach by introducing the continual learning (CL) framework. 
This framework enables the agents to incrementally learn and adapt to new environments while retaining the knowledge acquired from previous tasks <cit.>. By using this strategy, we aim to enhance the generalization capabilities of VLN agents, making them more robust and effective in real-world applications where they must navigate an ever-changing array of environments.


To enable the VLN agent to accumulate knowledge from tasks, handling the challenge known as catastrophic forgetting <cit.> is vital. We combine it with vision-language navigation tasks to introduce the Vision-Language Navigation with Continual Learning (VLNCL) paradigm. The agent must continuously accumulate information and maintain former knowledge by motivating the agent with new tasks. That means a balance between stability and plasticity <cit.>. Furthermore, considering that real-world tasks often occur within the same environment simultaneously, we split the tasks by scene to raise them to the agent. This way, tasks are divided into different domains.





Building on this foundation, we propose the dual-loop scenario replay vision-language navigation agent (Dual-SR) as a novel method for VLNCL. Inspired by the mechanism of memory replay in the resting brain <cit.>, we designed a dual-loop memory replay framework to enable the model to consolidate earlier scenario memories while balancing new task learning. Randomly replaying scenario memory from the memory buffer of the agent brings a former task memory bias while the inner loop weight updates learning new tasks <cit.>. 
Then, the agent applies the meta-learning-based outer loop weight updates to balance the new and old weights while alleviating the overfitting problem caused by the single scenario task data <cit.>.


Additionally, to effectively retain diverse environmental knowledge, we design a memory buffer based on task domains that allow agents to store and replay memories from different scenes.





To assess VLNCL agents, we propose two metrics: Unseen Transfer (UT) for evaluating knowledge transfer and Seen Transfer (ST) for assessing continual learning. UT measures generalization by testing the agent in new scenes with unseen tasks. ST evaluates continual learning by applying the agent to all seen task domains after training. 




Making the VLN agent able to perform continual learning in new tasks brings the advantage of increasing task performance and increasing task generalization. Comparative experiments were also conducted extensively with several CL methods used in other fields and previous VLN agents to verify the advances of our approach. Experiments show state-of-the-art performance in continual learning ability.


To summarize the contribution of this work:


    
    
    
  * We introduce the Vision-Language Navigation with Continual Learning (VLNCL) paradigm and metrics, enabling VLN agents to adapt to new environments while retaining prior knowledge for improving generalization. 
    
    
    
    
  * We propose Dual-loop Scenario Replay (Dual-SR), a novel approach inspired by the brain memory system, which enables continual learning in VLN agents. Experimental results showed a 16% success rate rise compared with the base model. 
    
    
    
  * We design a multi-scenario memory buffer that organizes task memories by environment type, facilitating rapid adaptation and balancing tasks across scenarios.
    
    
    
  * Our work focuses on continual learning in VLN agents, reduces catastrophic forgetting, and improves knowledge transfer, setting a benchmark in the field. Comparative experiments also confirm its robustness and effectiveness. 
    
    





§ RELATED WORK




 §.§ Vision-and-Language Navigation


Vision-and-Language Navigation (VLN) aims to develop intelligent agents capable of interacting with humans using natural language, perceiving the environment, and executing real-world tasks. This field has attracted significant attention across natural language processing, computer vision, robotics, and machine learning. Anderson et al. <cit.> laid the foundation by introducing the Room-to-Room (R2R) dataset, where agents navigate virtual indoor environments using a simulator. Building on R2R, researchers quickly established other VLN benchmarks like R4R <cit.> and RxR <cit.>.


For outdoor navigation, Touchdown <cit.> is a crucial benchmark where agents navigate a simulated New York City street view. Conversational navigation tasks, such as CVDN <cit.> and HANNA <cit.>, enable agents to interact with humans to aid navigation. Remote object navigation tasks, including REVERIE <cit.> and SOON <cit.>, require agents to infer object locations and identify them based on language instructions.

To address cross-modal alignment, PTA <cit.> leverages CNN-based visual features, while HAMT <cit.> incorporates long-term history into multimodal decision-making using a hierarchical vision transformer. Kerm <cit.> models relationships between scenes, objects, and directions using graphs. For navigation decisions, Dreamwalker <cit.> employs reinforcement learning with an intrinsic reward for cross-modal matching. Transformer-based models <cit.> has also gained popularity, integrating visual and linguistic information to enhance decision-making.

Despite these advances, introducing new environments requires retraining on seen environments to prevent catastrophic forgetting, leading to high costs and limited adaptability. Thus, enabling VLN agents with continual learning capabilities is crucial.














 §.§ Continual Learning





Continual learning is vital for agents to acquire new tasks without forgetting prior ones. Compared with traditional machine learning, which relies on static datasets, continual learning processes sequential data streams, making earlier data inaccessible. First introduced by Thrun <cit.>, recent research has focused on mitigating catastrophic forgetting <cit.>. Approaches in this field generally fall into three categories: rehearsal-based, regularization-based, and parameter isolation methods.






Rehearsal-based methods, like experience replay, retain data from previous tasks to update the model during new task training <cit.>. Techniques by Lopez-Paz <cit.> and Chaudhry <cit.> use data replay to prevent gradient conflicts. While combined with meta-learning and experience replay, these methods help optimize network features and reduce conflicts between tasks <cit.>. Capacity expansion approaches <cit.> and online meta-learning <cit.> offer additional strategies to mitigate forgetting.








In the VLN domain, continual learning research is still developing. Jeong et al. <cit.> recently proposed rehearsal-based methods like "PerpR" and "ESR" to enhance robots' adaptability in new environments, showing promise on specific datasets. However, these methods face memory and computational efficiency challenges, especially with complex, long-sequence tasks. Further refinement of the continual learning framework and evaluation metrics is needed.













§ METHOD




 §.§ Setting of Vision-Language Navigation


Vision-language navigation (VLN) involves an agent navigating a visual environment following natural language instructions. This task requires the agent to comprehend and integrate visual and linguistic inputs to achieve the goal. Let 𝐕 = {v_1, v_2, …, v_n} represent the sequence of visual observations from the environment and 𝐈 represent the natural language instruction. The objective is to learn a policy π(a_t |𝐕, 𝐈), where a_t is the action taken at time t, which maximizes the cumulative reward R = ∑_t=1^T r_t and minimize the loss 

    L=E_(I,A^*) ∈ D[ℒ(π_θ(a_t |𝐕, 𝐈), A^*)]

where r_t being the reward at time t, A^* is the ground truth trajectory, θ is parameters, and T the total number of time steps. 
The integration of vision and language occurs by projecting 𝐈 and 𝐕 into a common feature space, creating a joint embedding space.

This embedding allows the agent to align visual cues with linguistic references, enabling accurate navigation. 



 §.§ Formulation of Vision-Language Navigation with Continual Learning


Considering the reality application scenario, we take the validation dataset as a continual learning dataset and split it into several data streams. In real-world applications, the agent might navigate a scene for several tasks. Thus, we split the dataset by scene and divided the scene with the same ID into one data stream as one task domain, which agents learn sequentially. During this process, the agent needs to learn current tasks without forgetting knowledge of former tasks. This approach is more comparable to the realistic situations of agents in the real world. 

The dataset is separated into d task domains 𝒮={s_1, s_2, ..., s_d}, each containing tasks in the same scene. Task domains are independent of each other. Therefore, we can identify each task domain s as one distribution D_s. The definition of the loss function of VLNCL agents is

    min_θ∑^d_i=1 E_(I,A^*) ∈ D_s_i[ℒ(π_θ(a_t |𝐕, 𝐈), A^*)]

where parameters θ are iterated in each task domain. The agent can learn multiple task domains by optimizing the parameters in sequence. The main challenge of it is how to balance each task domain and promote generalization. 



 §.§ Dual-loop Scenario Replay



In the VLNCL setup, agents must minimize forgetting and improve transfer learning by leveraging prior knowledge to improve performance on current and former tasks. However, most existing VLN agents struggle with continual learning, unlike humans, who efficiently learn from a few examples by integrating sensory input with long-term memory <cit.>. The human brain continuously extracts and stores knowledge, reinforcing long-term memory through replay during rest <cit.>. Inspired by this, we propose the Dual-loop Scenario Replay Continual Learning (Dual-SR) algorithm for VLN agents. This algorithm simulates working memory as an inner loop and long-term memory as an outer loop, creating two weight update loops to balance prior and current task information while enhancing generalization.



In the VLNCL setup, agents might encounter an overfitting problem due to the limited samples. Hence, we leverage the meta-update mechanism in the Reptile algorithm <cit.> to mimic long-term memory formation. The reptile algorithm is equivalent in effectiveness to MAML <cit.>, which provides a means for models to acquire a standard structure from the current task domain, enabling them to quickly adapt to other similar new tasks. Thus, we can maximize generalization ability rather than data fitting. By that, the outer loop can improve the generalization capacity of agents. In the outer loop, the update of weight can be defined as:

    θ= θ + β*(θ^'-θ)

where θ, θ^', and β separately denote weights of the model before the inner cycle, weights after it, and meta-learning rate. 





To mimic the brain's abstraction and consolidation of long-term memories <cit.>, we designed the inner loop by simulating the memory retrieval in working memory. The agent can train with stable data selected randomly from previous task domains by maintaining the buffer and replaying old samples in the inner loop. Applying the memory buffer can ensure each former task is equally likely to be selected in the buffer. The newly received samples are combined with randomly selected old samples from the buffer to form a mini-batch, which is then used for meta-learning. 
In the inner loop, the update of the model can be defined as: 

    U_k(θ)=θ-α∘∇ L_(a_t |𝐕, 𝐈)(θ)

where U_k(θ) is a update by learning (a_t |𝐕, 𝐈) and α are parameters of the meta-learner to be learned, and ∘ denotes element-wise product. Specifically, α is a vector of the same size as θ that decides both the update direction and learning rates. 

Meta-updates can extract common structures learned across tasks, thereby enhancing the knowledge transfer capabilities of agents. The VLNCL setup exposes agents to a dynamic and unpredictable data stream. This procedure necessitates that agents adapt and perform effectively across an evolving array of tasks without following a predefined sequence. This approach differs significantly from traditional methods that require partitioning a fixed dataset into multiple batches for a set number of tasks. To address this, we implement experience replay within the inner loop. By storing task indices, the agent can revisit and leverage previously learned tasks when faced with new ones. 

This approach contrasts with traditional replay-based methods, which indiscriminately use memory across all tasks. Our method randomly replays scenarios within each task domain to ensure balance. Additionally, we introduce the memory buffer size Z. 
When a task belongs to a previous task domain and the task ID t is a multiple of Z, the agent updates the memory buffer M by replacing one of the tasks in the corresponding domain with the current task.

The agent efficiently manages memory size by updating scenario memory between task domains, even when handling many tasks. This strategy also encourages the model to prioritize tasks inspired by working memory principles.






























 §.§ Structured Transformer VLN Agents with Continual Learning


Building upon the Dual-SR algorithm, we employ a cross-modal structured transformer as the planner to enhance the performance of the VLN agent in continual learning settings. The Dual-SR algorithm provides the foundation for this approach by balancing integrating new information and retaining prior knowledge. At each navigation step t, the model processes five forms of tokens, global token 𝐠_t-1, candidate target tokens 𝐂 = {𝐜_t-1^1, 𝐜_t-1^2, …, 𝐜_t-1^q}, history tokens 𝐇 = {𝐡_t-1^1, 𝐡_t-1^2, …, 𝐡_t-1^t-1}, encoded instruction tokens 𝐈 = {𝐢_0, 𝐢_1, …, 𝐢_m}, and encoded vision tokens 𝐕 = {𝐯_t^1, 𝐯_t^2, …, 𝐯_t^n}. The instruction tokens remain constant through time to reduce computation, and other tokens are updated based on previous time steps. The system initializes the global token as the sentence embedding 𝐠_0 = i_0.




To encode candidate target tokens, we apply the grid coding form to address the modeling of possible long-term target challenges in unseen scenes. Each cell center can represent a potential navigation target token by discretizing the environment into a d × d grid to cover the navigation area. Initially, candidate target tokens 𝐜_0^1, 𝐜_0^2, …, 𝐜_0^q are created using the positional embedding of the targets, formulated as:

    𝐜_0^i = f_ϕ_P(𝐬_j) ·𝐢_0,    j ∈{1, 2, …, q}

where f_P is the positional encoder, 𝐬_i is the spatial location expressed by position coordinates, ϕ_P is the parameter of encoder, and 𝐱_0 is the sentence embedding.







During navigation, these candidate target tokens are refined with new visual clues and instruction tokens to predict a more precise long-term target. The probability of each target being the navigation destination is calculated using a multi-layer perceptron (MLP) based target predictor:

    0.9!P(𝐜_t^i |θ) = softmax{MLP(𝐜_t^i ·𝐠_t)},    i ∈{1, 2, …, q}

where 𝐠_t is the global token.







The agent constructs and maintains a structured representation of the explored area with the transformer architecture to capture the structured environment layouts. At time step t, the model constructs a graph 𝒮_t, where the nodes represent previously visited locations and the edges represent the navigability of those locations. We construct the history token 𝐡_t^t using panoramic view embedding, action embedding, temporal embedding, and positional embedding as follows:


    𝐡_t^t = f_V(𝐯_t^1, …, 𝐯_t^n) + f_A(𝐫_t) + f_T(t) + f_P(𝐬_t)

where f_V is a panoramic visual feature extractor, 𝐫_t = (sinθ, cosθ, sinφ, cosφ) is the moving direction, f_A is the action encoder, f_T is the temporal encoder, and f_P is the positional encoder.







The adjacency matrix 𝐄 of history tokens at time step t is defined such that if a navigation viewpoint n_j is navigable from n_i, then 𝐄_ij = 1; otherwise, 𝐄_ij = 0. The attention mask matrix 𝐌 controls the information flow among tokens, with a sub-matrix 𝐌_H for history tokens:

    𝐌_H ←𝐌_H ∗𝐄

where ∗ denotes element-wise multiplication.













The structured transformer enables the agent to access structured information of the past, allowing decisions from adjacent and previously visited locations. The local action space at time step t is:

    𝒜_t^L = {τ(𝐯̂_t^1), τ(𝐯̂_t^2), …, τ(𝐯̂_t^k_t)}

and the global action space is:

    𝒜_t^G = {τ(𝐯̂_t^1), …, τ(𝐯̂_t^k_t), τ(𝐡_t^1), …, τ(𝐡_t^t-1)}

where τ maps the token to its corresponding location. The probability of each possible action is:

    0.87!π(a_t |θ) = softmax{MLP(τ^-1(a_t) ·𝐠_t)},    a_t ∈𝒜_t^G


The optimization of the model involves both imitation learning (IL) loss L_IL and reinforcement learning (RL) loss L_RL, alternating between teacher forcing (using ground truth actions) and student forcing (using actions sampled from the policy). To further consider the action chosen and target chosen, the history teacher loss L_HT and target prediction loss L_T are incorporated.
The history teacher loss is defined as:

    L_HT = -∑_t=1^T logπ(a_t |θ)

and the target prediction loss is:

    L_T = -∑_t=1^T log P(𝐜_t^i |θ)

where the i-th target token is closest to the navigation destination.
The total loss function is given by:

    L = α_1 L_IL + α_2 L_RL + α_3 L_HT + α_4 L_T

where α_i are the loss coefficients.

After training the foundational model, the agent prompts continual learning inference within validation environments. The agent processes the task-domain-based data stream sequentially for the Val-Seen and Val-Unseen splits. The agent executes the inner loop based on the loss function specified in Equation <ref> to iteratively update parameters θ, achieved through continuous memory buffer updates and scenario replays. Upon completing the learning for the current task domain, the agent proceeds to perform the outer loop as described in Equation <ref>.



The continual learning methodology equips the VLN agent to learn and adapt within complex environments, maintaining and enhancing knowledge across multiple tasks. The Dual-SR algorithm in structured transformers allows for effective navigation and adaptation capabilities in continually changing scenarios.



























§ EXPERIMENTS

























 §.§ Experiment Setup



The experiment adopts the VLNCL framework and divides the R2R dataset <cit.> into distinct task domains to evaluate resistance to forgetting and knowledge transfer capabilities.
By sequentially inputting each task domain into the agent, we separately assess the average Seen Transfer (ST) and Unseen Transfer (UT) across each dataset split.



For comparative experiments, we implemented the latest unpublished related work with the corresponding setup and a prior study incorporating continual learning. We also compared our approach with other VLN agents to validate performance on the test unseen split and confirm that continual learning enhances agent performance.



 §.§ Evaluation Protocol for VLNCL





To assess the forgetting resistance and knowledge transfer abilities of the VLNCL agent, we introduce two metrics: Seen Transfer (ST) and Unseen Transfer (UT), analogous to Backward and Forward Transfer in continual learning. ST measures the average performance difference on the i-th task domain after training on T domains compared to training on the first i domains (i < T). UT evaluates the average performance difference on unseen tasks after training on T domains compared to the base agent. As training progresses, unseen tasks diminish as the agent encounters them. 
Note that the unseen task domain set is a subset of the validation task set, divided into several scenario-related data streams that feed the agent sequentially. Therefore, the unseen task domain set is dynamic.


Formally, the validation task set is separated into d task domains 𝒮={s_1, s_2, ..., s_d}. The initial unseen task domain set is 𝒮_unseen^0={s_1, s_2, ..., s_d}. After the agent encounters task domain s_1, the unseen task domain set is updated with 𝒮_unseen^1={s_2, s_3, ..., s_d}. Hence, the Seen Transfer is defined as

    ST = 1/T-1∑_i=1^T-1 SR_T(s_i) - SR_i(s_i)    s_i ∈𝒮_seen^T

The Unseen Transfer is defined as

    0.85!UT = 1/T-1∑_i=2^T SR_i-1(s_i) - SR_0(s_i)    s_i ∈𝒮_unseen^T




where the SR_j(s_i) is the average success rate in the task domain s_i for agent after trained on the j task domain. The SR_0(s_i) represents the success rate of the base agent.



 §.§ Implementation Details






The agent's hyperparameters and the structured transformer architecture align with those utilized in previous studies. The candidate target grid size, d, is set to 5, resulting in a 5 × 5 grid with a spacing of 6 meters between adjacent positions. The agent follows the ground-truth path when applying teacher force under specific loss configurations. Conversely, actions are sampled from the predicted probability distribution during student forcing, leveraging distinct loss configurations. 

We train the base model using the Adam optimizer on two NVIDIA V100 GPUs for 100,000 iterations, employing a batch size 20 and a learning rate of 1×10^-5 over 72 GPU hours.
Continual learning is subsequently conducted on two NVIDIA V100 GPUs for 1k iterations per task domain with learnable learning rates.



 §.§ Comparative Experiment Results





To evaluate the performance of our continual learning method in vision-language navigation tasks, we conducted a comparative experiment against previous works using the R2R dataset. We adopted the latest VLN agents with continual learning capabilities, adhering to the experimental settings and metrics outlined in ESR <cit.>, which evaluate the average performance across all seen tasks throughout the continual learning process. Additionally, we compared our framework with the ASA continual learning strategy <cit.>, which fine-tunes the agent using new data. However, distinguished from the original ASA, our experiment employed augmented data rather than interaction data collected from human-agent interactions. Agents used a randomly initialized HAMT <cit.> as the backbone model and set the agent's buffer size to 2000 tasks. Due to GPU memory constraints, the agent stored only each task's index and environment ID. The baseline for the experiment was the agent that did not employ any continual learning algorithm.





To further evaluate the task performance of our agent, we compared its results on the test unseen split against single-run performances of other VLN agents on the R2R dataset. The methods included in this comparison are Seq2Seq <cit.>, SSM <cit.>, EnvDrop <cit.>, AuxRN <cit.>, CCC <cit.>, PREVALENT <cit.>, AirBERT <cit.>, VLN↻BERT <cit.> (init. OSCAR). 
The baseline used the base agent's performance before applying continual learning. We further compared the performance before and after continuous learning to demonstrate the potential of CL approaches.


 
 
 
 
 
 
 
 
 
 
 








The comparative results demonstrate that our method significantly enhances the task performance of the agent. The success rate increased by 16%, and the oracle success rate increased by 8% compared to the base agent, highlighting the substantial potential of a continual learning approach in VLN agents. Moreover, our results have achieved state-of-the-art performance in continual learning ability. Unlike agents that rely on elaborate environmental understanding mechanisms or meticulous fine-tuning, agents with CL capabilities can consistently enhance their performance in new scenes. However, integrating a more advanced inference architecture with the CL approach can significantly improve the agent's performance.






 §.§ Resisting Forgetting and Transferring Evaluation








With continual learning, we introduce the concepts of Seen Transfer and Unseen Transfer for the VLN agent to evaluate its resistance to forgetting and ability to transfer knowledge. To assess forgetting resistance, we apply the average Seen Transfer to the validation train split and the validation seen split. We utilize the average Unseen Transfer on the validation unseen split to evaluate the agent's capability to transfer knowledge. As a baseline, we use the results of fine-tuning each task domain.

The Seen Transfer measures the agent's performance on the seen task domain set, 𝒮_seen^T, within the current split, determining whether the agent retains its prior knowledge. In contrast, Unseen Transfer evaluates performance on the unseen task domain set, 𝒮_unseen^T, within the current split, assessing knowledge transfer from previous tasks to current ones.





The experimental results demonstrated that our method has advantages in forgetting resistance and knowledge transfer. To further elucidate these capabilities, we present the changes in success rates across different splits in Figure <ref>. In the Val Unseen split, we evaluate the success rate in the unseen portion to assess knowledge transfer ability. Conversely, in the Val Seen and Train Seen splits, we consider the success rate in the seen portion to evaluate the agent's forgetting resistance.



Additionally, we can track the highest and lowest Success Rates (SR) and Oracle Success Rates (OSR) across all dataset splits in Table <ref>. The evaluation of performance also follows the VLNCL setup. This result allows us to observe the changes in performance more clearly.

































Analyzing performance changes shows that our method exhibits strong resistance to forgetting and robust knowledge transfer capabilities. Our method consistently improves performance on unseen tasks in the Val Unseen split, highlighting the significant generalization capability introduced by continual learning in VLN agents. In the Val Seen and Train Seen splits, our method also shows substantial performance retention on seen tasks, with a success rate drop of only 20% after processing a sequence of 50 different scenario tasks. These results indicate that the agent effectively mitigates forgetting across long-term task domains. Consequently, we propose a benchmark for the Vision-Language Navigation with Continual Learning paradigm. 

By implementing a continual learning approach in VLN agents, we enhance their ability to generalize effectively to unseen environments, paving the way for broader real-world applications of VLN agents.





§ CONCLUSION







This paper presents the Vision-Language Navigation with Continual Learning (VLNCL) paradigm, where the agent learns from unseen tasks while retaining knowledge from prior scenarios, closely reflecting real-world application demands. To achieve this, we introduce the Dual-loop Scenario Replay (Dual-SR) algorithm, which improves the agent's generalization and task performance. We also establish a benchmark for VLNCL using the R2R dataset. Experiments demonstrate that our method surpasses existing continual learning approaches under comparable conditions, advancing VLN agent performance and setting the stage for further research into real-world application-ready agents. Our future work will concentrate on developing more sophisticated inference mechanisms and advancing continual learning strategies to improve generalization.














