Skip to content

Commit

Permalink
Update to use SK 1.0.1 and update version of plugin to 1.0.3
Browse files Browse the repository at this point in the history
  • Loading branch information
Swimburger committed Dec 20, 2023
1 parent 2264f0b commit 4a3f949
Show file tree
Hide file tree
Showing 6 changed files with 72 additions and 76 deletions.
57 changes: 31 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,14 @@ using AssemblyAI.SemanticKernel;
using Microsoft.SemanticKernel;

// Build your kernel
var kernel = new KernelBuilder().Build();
var kernel = Kernel.CreateBuilder();

// Get AssemblyAI API key from env variables, or much better, from .NET configuration
string apiKey = Environment.GetEnvironmentVariable("ASSEMBLYAI_API_KEY")
?? throw new Exception("ASSEMBLYAI_API_KEY env variable not configured.");

kernel.ImportFunctions(
new TranscriptPlugin(apiKey: apiKey),
kernel.ImportPluginFromObject(
new TranscriptPlugin(apiKey: apiKey)
TranscriptPlugin.PluginName
);
```
Expand All @@ -45,12 +45,16 @@ kernel.ImportFunctions(

Get the `Transcribe` function from the transcript plugin and invoke it with the context variables.
```csharp
var function = kernel.Functions
.GetFunction(TranscriptPlugin.PluginName, TranscriptPlugin.TranscribeFunctionName);
var context = kernel.CreateNewContext();
context.Variables["INPUT"] = "https://storage.googleapis.com/aai-docs-samples/espn.m4a";
var result = await function.InvokeAsync(context);
Console.WriteLine(result.GetValue<string());
var arguments = new KernelArguments
{
["INPUT"] = "https://storage.googleapis.com/aai-docs-samples/espn.m4a"
};
var result = await kernel.InvokeAsync(
TranscriptPlugin.PluginName,
TranscriptPlugin.TranscribeFunctionName,
arguments
);
Console.WriteLine(result.GetValue<string>());
```

You can get the transcript using `result.GetValue<string>()`.
Expand All @@ -60,42 +64,43 @@ You can also upload local audio and video file. To do this:
- Configure the `INPUT` variable with a local file path.

```csharp
kernel.ImportFunctions(
kernel.ImportPluginFromObject(
new TranscriptPlugin(apiKey: apiKey)
{
AllowFileSystemAccess = true
},
TranscriptPlugin.PluginName
);
var function = kernel.Functions
.GetFunction(TranscriptPlugin.PluginName, TranscriptPlugin.TranscribeFunctionName);
var context = kernel.CreateNewContext();
context.Variables["INPUT"] = "./espn.m4a";
var result = await function.InvokeAsync(context);
var arguments = new KernelArguments
{
["INPUT"] = "https://storage.googleapis.com/aai-docs-samples/espn.m4a"
};
var result = await kernel.InvokeAsync(
TranscriptPlugin.PluginName,
TranscriptPlugin.TranscribeFunctionName,
arguments
);
Console.WriteLine(result.GetValue<string>());
```

You can also invoke the function from within a semantic function like this.

```csharp
var prompt = """
Here is a transcript:
{{TranscriptPlugin.Transcribe "https://storage.googleapis.com/aai-docs-samples/espn.m4a"}}
---
Summarize the transcript.
""";
var context = kernel.CreateNewContext();
var function = kernel.CreateSemanticFunction(prompt);
var result = await function.InvokeAsync(context);
const string prompt = """
Here is a transcript:
{{TranscriptPlugin.Transcribe "https://storage.googleapis.com/aai-docs-samples/espn.m4a"}}
---
Summarize the transcript.
""";
var result = await kernel.InvokePromptAsync(prompt);
Console.WriteLine(result.GetValue<string>());
```

All the code above explicitly invokes the transcript plugin, but it can also be invoked as part of a plan.
Check out [the Sample project](./src/Sample/Program.cs#L96)) which uses a plan to transcribe an audio file in addition to explicit invocation.
Check out [the Sample project](./src/Sample/Program.cs#L87)) which uses a plan to transcribe an audio file in addition to explicit invocation.

## Notes

- The AssemblyAI integration only supports Semantic Kernel with .NET at this moment.
If there's demand, we will extend support to other platforms, so let us know!
- Semantic Kernel itself is still in pre-release, and changes frequently, so we'll keep our integration in pre-release until SK is GA'd.
- Feel free to [file an issue](https://github.com/AssemblyAI/assemblyai-semantic-kernel/issues) in case of bugs or feature requests.
10 changes: 5 additions & 5 deletions src/AssemblyAI.SemanticKernel/AssemblyAI.SemanticKernel.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@
<PackageTags>SemanticKernel;AI;AssemblyAI;transcript</PackageTags>
<Company>AssemblyAI</Company>
<Product>AssemblyAI</Product>
<AssemblyVersion>1.0.2.0</AssemblyVersion>
<FileVersion>1.0.2.0</FileVersion>
<PackageVersion>1.0.2-alpha</PackageVersion>
<AssemblyVersion>1.0.3.0</AssemblyVersion>
<FileVersion>1.0.3.0</FileVersion>
<PackageVersion>1.0.3</PackageVersion>
<OutputType>Library</OutputType>
<PackageLicenseExpression>MIT</PackageLicenseExpression>
<PackageProjectUrl>https://github.com/AssemblyAI/assemblyai-semantic-kernel</PackageProjectUrl>
Expand All @@ -32,12 +32,12 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.SemanticKernel">
<Version>1.0.0-beta8</Version>
<Version>1.0.1</Version>
</PackageReference>
<PackageReference Include="System.Net.Http.Json">
<Version>8.0.0</Version>
</PackageReference>
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.1.1">
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
Expand Down
2 changes: 1 addition & 1 deletion src/AssemblyAI.SemanticKernel/TranscriptPlugin.cs
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ public TranscriptPlugin(string apiKey)

public const string TranscribeFunctionName = nameof(Transcribe);

[SKFunction, Description("Transcribe an audio or video file to text.")]
[KernelFunction, Description("Transcribe an audio or video file to text.")]
public async Task<string> Transcribe(
[Description("The public URL or the local path of the audio or video file to transcribe.")]
string input
Expand Down
20 changes: 6 additions & 14 deletions src/Sample/FindFilePlugin.cs
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,13 @@ namespace AssemblyAI.SemanticKernel.Sample;
public class FindFilePlugin
{
public const string PluginName = nameof(FindFilePlugin);
private readonly IKernel _kernel;


public FindFilePlugin(IKernel kernel)
{
_kernel = kernel;
}

private async Task<string?> GetCommonFolderPath(string commonFolderName)
private async Task<string?> GetCommonFolderPath(Kernel kernel, string commonFolderName)
{
var prompt = $"The path for the common folder '{commonFolderName}' " +
$"on operating platform {Environment.OSVersion.Platform.ToString()} " +
$"with user profile path '{Environment.GetFolderPath(Environment.SpecialFolder.UserProfile)}' is ";
var context = await _kernel.InvokeSemanticFunctionAsync(
template: prompt
);
var context = await kernel.InvokePromptAsync(prompt);
var matches = Regex.Matches(
context.GetValue<string>()!,
@"([a-zA-Z]?\:?[\/\\][\S-[""'\. ]]*[\/\\][\S-[""'\. ]]*)",
Expand All @@ -31,11 +22,12 @@ public FindFilePlugin(IKernel kernel)
return matches.LastOrDefault()?.Value ?? null;
}

[SKFunction, Description("Find files in common folders.")]
[KernelFunction, Description("Find files in common folders.")]
public async Task<string> LocateFile(
[Description("The name of the file")] string fileName,
[Description("The name of the common folder")]
string? commonFolderName)
string? commonFolderName,
Kernel kernel)
{
var commonFolderPath = commonFolderName?.ToLower() switch
{
Expand All @@ -48,7 +40,7 @@ public async Task<string> LocateFile(
"pictures" => Environment.GetFolderPath(Environment.SpecialFolder.MyPictures),
"documents" => Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments),
"user" => Environment.GetFolderPath(Environment.SpecialFolder.UserProfile),
_ => await GetCommonFolderPath(commonFolderName)
_ => await GetCommonFolderPath(kernel, commonFolderName)
?? throw new Exception("Could not figure out the location of the common folder.")
};

Expand Down
52 changes: 24 additions & 28 deletions src/Sample/Program.cs
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
using System.Text.Json;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Planners;
using Microsoft.SemanticKernel.Planning.Handlebars;

namespace AssemblyAI.SemanticKernel.Sample;

Expand All @@ -14,36 +13,31 @@ public static async Task Main(string[] args)

var kernel = BuildKernel(config);

await TranscribeFileUsingPluginDirectly(kernel);
await TranscribeFileUsingPluginFromSemanticFunction(kernel);
//await TranscribeFileUsingPluginDirectly(kernel);
//await TranscribeFileUsingPluginFromSemanticFunction(kernel);
await TranscribeFileUsingPlan(kernel);
}

private static IKernel BuildKernel(IConfiguration config)
private static Kernel BuildKernel(IConfiguration config)
{
var loggerFactory = LoggerFactory.Create(builder => { builder.SetMinimumLevel(0); });
var kernel = new KernelBuilder()
.WithOpenAIChatCompletionService(
var kernel = Kernel.CreateBuilder()
.AddOpenAIChatCompletion(
"gpt-3.5-turbo",
config["OpenAI:ApiKey"] ?? throw new Exception("OpenAI:ApiKey configuration is required.")
)
.WithLoggerFactory(loggerFactory)
.Build();

var apiKey = config["AssemblyAI:ApiKey"] ?? throw new Exception("AssemblyAI:ApiKey configuration is required.");

kernel.ImportFunctions(
kernel.ImportPluginFromObject(
new TranscriptPlugin(apiKey: apiKey)
{
AllowFileSystemAccess = true
},
TranscriptPlugin.PluginName
);

kernel.ImportFunctions(
new FindFilePlugin(kernel),
FindFilePlugin.PluginName
);
kernel.ImportPluginFromType<FindFilePlugin>(FindFilePlugin.PluginName);
return kernel;
}

Expand All @@ -57,20 +51,24 @@ private static IConfigurationRoot BuildConfig(string[] args)
return config;
}

private static async Task TranscribeFileUsingPluginDirectly(IKernel kernel)
private static async Task TranscribeFileUsingPluginDirectly(Kernel kernel)
{
Console.WriteLine("Transcribing file using plugin directly");
var context = kernel.CreateNewContext();
context.Variables["INPUT"] = "https://storage.googleapis.com/aai-docs-samples/espn.m4a";
var result = await kernel.Functions
.GetFunction(TranscriptPlugin.PluginName, TranscriptPlugin.TranscribeFunctionName)
.InvokeAsync(context);
var arguments = new KernelArguments
{
["INPUT"] = "https://storage.googleapis.com/aai-docs-samples/espn.m4a"
};
var result = await kernel.InvokeAsync(
TranscriptPlugin.PluginName,
TranscriptPlugin.TranscribeFunctionName,
arguments
);

Console.WriteLine(result.GetValue<string>());
Console.WriteLine();
}

private static async Task TranscribeFileUsingPluginFromSemanticFunction(IKernel kernel)
private static async Task TranscribeFileUsingPluginFromSemanticFunction(Kernel kernel)
{
Console.WriteLine("Transcribing file and summarizing from within a semantic function");
// This will pass the URL to the `INPUT` variable.
Expand All @@ -81,24 +79,22 @@ private static async Task TranscribeFileUsingPluginFromSemanticFunction(IKernel
---
Summarize the transcript.
""";
var context = kernel.CreateNewContext();
var function = kernel.CreateSemanticFunction(prompt);
var result = await function.InvokeAsync(context);
var result = await kernel.InvokePromptAsync(prompt);
Console.WriteLine(result.GetValue<string>());
Console.WriteLine();
}

private static async Task TranscribeFileUsingPlan(IKernel kernel)
private static async Task TranscribeFileUsingPlan(Kernel kernel)
{
Console.WriteLine("Transcribing file from a plan");
var planner = new SequentialPlanner(kernel);
var planner = new HandlebarsPlanner(new HandlebarsPlannerOptions { AllowLoops = true });
const string prompt = "Find the espn.m4a in my downloads folder and transcribe it.";
var plan = await planner.CreatePlanAsync(prompt);
var plan = await planner.CreatePlanAsync(kernel, prompt);

Console.WriteLine("Plan:\n");
Console.WriteLine(JsonSerializer.Serialize(plan, new JsonSerializerOptions { WriteIndented = true }));

var transcript = (await kernel.RunAsync(plan)).GetValue<string>();
var transcript = await plan.InvokeAsync(kernel);
Console.WriteLine(transcript);
Console.WriteLine();
}
Expand Down
7 changes: 5 additions & 2 deletions src/Sample/Sample.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<UserSecretsId>9bf031e1-85d9-44ec-bcc3-bcf25b26eeaa</UserSecretsId>
<RootNamespace>AssemblyAI.SemanticKernel.Sample</RootNamespace>
<NoWarn>SKEXP0060</NoWarn>
</PropertyGroup>

<ItemGroup>
Expand All @@ -33,7 +33,10 @@
<Version>8.0.0</Version>
</PackageReference>
<PackageReference Include="Microsoft.SemanticKernel">
<Version>1.0.0-beta8</Version>
<Version>1.0.1</Version>
</PackageReference>
<PackageReference Include="Microsoft.SemanticKernel.Planners.Handlebars">
<Version>1.0.1-preview</Version>
</PackageReference>
</ItemGroup>
</Project>

0 comments on commit 4a3f949

Please sign in to comment.