Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Local UE Generation Handler Due to External Kafka Issues #10

Closed
wants to merge 21 commits into from
Closed

Implement Local UE Generation Handler Due to External Kafka Issues #10

wants to merge 21 commits into from

Conversation

tanzim10
Copy link
Contributor

Overview

This PR introduces a local User Equipment (UE) generation handler to replace the current API service, which is failing due to unresolved Kafka connectivity issues. The new local handler ensures continued functionality of UE simulations without external dependencies.

Changes Made

  • Local UE Generation Classes: Developed custom classes that handle the generation of UE data locally.
  • Mobility Data Generation: This function generates UE mobility data by converting XY coordinates to longitude and latitude, organizing the data into a pandas DataFrame for each simulation tick across batches.
  • Data Output: Implemented functionality to plot simulation results and save the data directly to CSV format for further analysis.

Justification

The external API service for UE generation was reliant on Kafka streams, which are currently facing connectivity issues. To maintain productivity and ensure the reliability of our simulations, switching to a local generation method was deemed necessary.

Copy link

linux-foundation-easycla bot commented Sep 15, 2024

CLA Not Signed

Copy link
Collaborator

@paulvarkey paulvarkey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR may also have a notebook that uses the API improvements here.




def mobility_data_generation(self) -> pd.DataFrame:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this method and generate_ue_data can be replaced with the original _mobility_data_generation, which can actually be moved to this directory (as you are correctly doing in this PR), into ue_tracks.py, and we may rename it as generate_as_lon_lat_points, and then fix the call-site.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the ue_tracks by adding _mobility_data_generation from services/ue_tracks_generation/u_tracks_generation_driver.py file to ue_tracks.py.
Renamed it to generate_as_lon_lat_points.

Will delete the reduntant ue_generation_handler.py after plot functions is fixed.

return pd.concat(all_dataframes, ignore_index=True)


def plot_ue_tracks(self,csv_file) -> None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make this a static method inside ue_tracks that takes a UETracksGenerator object and simulates and plots it. The whole simulation can be done and the whole data can be loaded into memory and plotted (as is being done here) OR, if possible, just enumerate through generate_as_lon_lat_points and plot one by one.

# Update start_idx for the next batch
start_idx = end_idx

def save_data_to_csv(self, filename: str) -> None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you may delete this (in future, you may provide such a utility if we need to build in more business logic around this export)

Copy link
Collaborator

@paulvarkey paulvarkey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve -- please add test_ue_tracks_params.py

)


class UETracksGenerationParams:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a unit test

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the Unit Test Cases for the ue_tracks_params.
All of the changes are made.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You check the second last commit to see the changes made in mobility/test module. The last commit was done to format all of the codes using Black.

tanzim10 and others added 3 commits September 24, 2024 11:30
* changed location for handler and added unit tests

* fixed hanlder position

* fixed import locations

* used black on test & helper files

* fixed name and removed redundant code in test

* added docstring

* added HOW TO RUN comment

* blacked

* fixed typo

* made pr comment changes

* added eu-tracks-params test code

* fixed test_ue_params

* added docstring

* changed venv issues
@paulvarkey
Copy link
Collaborator

Closing since done in #11

@paulvarkey paulvarkey closed this Oct 1, 2024
@tanzim10 tanzim10 deleted the ue-generation-handler branch October 1, 2024 05:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants