Out my depth (hashing data)...could do with some help!

[peskywinnets]

Ok, so I'm trying to knock up an API to connect to Amazon...the API itself isn't the problem, it's the hashing process.

To test everyone's hashing process, Amazon provided some test data to work with.

The process is to start off with a known key, then use that key to hash some data .....the output of which you then use to hash the next piece of data...and so on

so we start off with a signing key of AWS4wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY ...the first thing we want to hash is a sample date of 20120215 therefore

kdate = ComputeHMACSHA256("AWS4wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY", "20120215")

that should produce an output of 969fbb94feb542b71ede6f87fe4d5fa29c789342b0f407474670f0c2489e0a0d ...and this bit does indeed work.

we then use the output to hash another bit of sample region data us-east-1 therefore...

kregion = ComputeHMACSHA256("969fbb94feb542b71ede6f87fe4d5fa29c789342b0f407474670f0c2489e0a0d", "us-east-1")


that should produce an output of 69daa0209cd9c5ff5c8ced464a696fd4252e981430b10e3d3fd8e2f197d7a70c ...but it doesn't

Someone else raised this here...

AWS4 Signature key - is this tutorial wrong?

According to this page: Examples of How to Derive a Signing Key for Signature Version 4 The result of this code: $kSecret = "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY"; $kDate = hash_hmac('sha256...

stackoverflow.com

....and someone did chime in with an answer & the associated code (which produces the correct output), but they are discussing PHP - I'm out my depth and don't know how to deploy in VBA (on stackoverflow, they reference an online hashing tool... https://hash.online-convert.com/sha256-generator )

It seems to hinge on how data is stored vs how it's displayed ...but I've read it way too many times & I'm not getting it.

Here's the simple code if anyone wants a dabble....

Public Function TestHMAC()
Dim kdate As String
Dim kregion As String

Key = "AWS4wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY"
dateStamp = "20120215"
regionName = "us-east-1"


kdate = ComputeHMACSHA256("AWS4wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY", "20120215")
Debug.Print kdate  ' should come out as    969fbb94feb542b71ede6f87fe4d5fa29c789342b0f407474670f0c2489e0a0d
kregion = ComputeHMACSHA256(kdate, "20120215")
End Function

Function ComputeHMACSHA256(Key As String, Text As String) As String
  Dim encoder As Object, crypto As Object
  Dim hash() As Byte, hmacsha As String, i As Long

  ' compute HMACSHA256
  Set encoder = CreateObject("System.Text.UTF8Encoding")
  Set crypto = CreateObject("System.Security.Cryptography.HMACSHA256")
  crypto.Key = encoder.GetBytes_4(Key)
  hash = crypto.ComputeHash_2(encoder.GetBytes_4(Text))

  ' convert to an hexa string
  hmacsha = String(64, "0")
  For i = 0 To 31
     Mid$(hmacsha, i + i + (hash(i) > 15) + 2) = Hex(hash(i))
  Next

ComputeHMACSHA256 = LCase(hmacsha)
End Function

 

[theDBguy]

Whew! I don't remember it being this complicated when I did it before.

 

[isladogs]

For info, I've asked an encryption expert from another forum to have a look at this

 

[peskywinnets]

Amazon have changed their signing process...before was Signing Version 2 ... they are now deploying Signing Version 4 - way more convoluted.

Like most things if you break the overall large chunks down into smaller components it becomes digestible ...but I'm failing on this one (& it all hinges of the string sent to Amazon being Signed correctly!), looking at the solution on stackflow (in python) as a likely solution to the same test problem being posed on stackoverflow)...

import hmac
import hashlib
from base64 import b16encode as b16

def sign(key, msg):
    return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    kDate = sign(("AWS4" + key).encode("utf-8"), dateStamp)
    print b16(kDate)
    kRegion = sign(kDate, regionName)
    print b16(kRegion)
    kService = sign(kRegion, serviceName)
    print b16(kService)
    kSigning = sign(kService, "aws4_request")
    print b16(kSigning)
    return kSigning

key = 'wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY'
dateStamp = '20120215'
regionName = 'us-east-1'
serviceName = 'iam'

getSignatureKey(key, dateStamp, regionName, serviceName)

Is anyone python literate? becuase therein lies my solution .....if I can suss what is going on & get the same results ....& then do similar VBA ? (I downloaded Thonny (a simple Python interface), but the above code is failing on... print b16(kDate) ...the very bit that seems pivotal

 

[theDBguy]

Amazon have changed their signing process...before was Signing Version 2 ... they are now deploying Signing Version 4 - way more convoluted.

That's it. I just looked at my old code, and it does say Version=2. However, I wonder if it still work, because I haven't heard from them asking me to update it.

 

[peskywinnets]

It will still work (I still have some old APIs running on their Signing V2)... Signing V4 is enforced only for later APIs

 

[isladogs]

I expect it has been updated (made more complicated) to comply with the latest data protection regulations

 

[peskywinnets]

For info, I've asked an encryption expert from another forum to have a look at this

That's great, for the purposes of signing, I'm looking at the section called "Deriving a signing key using other languages", at the foot of this page...

Create a signed AWS API request - AWS Identity and Access Management

Learn how to use the AWS SigV4 signing protocol to create a signed request for AWS API requests. You’ll learn how to create and hash a canonical request, create a string to sign, derive a signing key, and calculate a signature to add to the request.

docs.aws.amazon.com

....it has the test data & results - I seek to sign that test/sample data & end up with the same results using VBA (if you scroll up the page, there are examples of how to do this in other languages)

 

[isladogs]

OK. No promises as to whether they will respond. I'm not the expert

 

[peskywinnets]

Ok, I've managed to get this bit working...full kudos & credit to Trentium over on stackoverflow who provided this VBA code

https://stackoverflow.com/questions/50786635/integrating-amazon-polly-with-vba

I've picked the bones out of his code (i.e. the bits I need for hashing) & thankfully have ended up with the same hash result as the data on amazon's sample/test page

here are the key bits

Public Function HashTest()
Call getSignatureKey("AWS4wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY", "20120215", "us-east-1", "iam")
End Function

Private Function getSignatureKey(key As String, dateStamp As String, regionName As String, serviceName As String) As Byte()

    Dim kDate() As Byte, kRegion() As Byte, kService() As Byte, kSigning() As Byte
    kDate = hmacSha256(key, dateStamp)
    kRegion = hmacSha256(kDate, regionName)
    kService = hmacSha256(kRegion, serviceName)
    kSigning = hmacSha256(kService, "aws4_request")

    getSignatureKey = kSigning

    Debug.Print byte2hex(getSignatureKey)   '<< this is the key bit, it gives the hash result  f4780e2d9f65fa895f9c67b32ce1baf0b0d8a43505a000a1a9e090d414db404d ...which is what it should be   :-)

End Function
Public Function hmacSha256(key As Variant, stringToHash As Variant) As Byte()

    Dim ssc As Object
    Set ssc = CreateObject("System.Security.Cryptography.HMACSHA256")

    ssc.key = str2byte(key)
    hmacSha256 = ssc.ComputeHash_2(str2byte(stringToHash))

    Set ssc = Nothing

End Function
Public Function sha256(stringToHash As Variant) As Byte()

    Dim ssc As Object
    Set ssc = CreateObject("System.Security.Cryptography.SHA256Managed")

    sha256 = ssc.ComputeHash_2(str2byte(stringToHash))

    Set ssc = Nothing

End Function
Public Function str2byte(s As Variant) As Byte()

    If VarType(s) = vbArray + vbByte Then
        str2byte = s
    ElseIf VarType(s) = vbString Then
        str2byte = StrConv(s, vbFromUnicode)
    Else
        Exit Function
    End If

End Function
Public Function byte2hex(byteArray() As Byte) As String

    Dim i As Long
    For i = 0 To UBound(byteArray)
        byte2hex = byte2hex & Right(Hex(256 Or byteArray(i)), 2)
    Next
    byte2hex = LCase(byte2hex)

End Function

....this should get me moving along, until I come unstuck at the next point!

Many thanks for lending an ear & the offer of help (I've been on this a few nights - typical isn't it? i.e., as soon as posing the question, my mate Google solves it within a few hours!)

 

[theDBguy]

Ok, I've managed to get this bit working...full kudos & credit to Trentium over on stackoverflow who provided this VBA code

https://stackoverflow.com/questions/50786635/integrating-amazon-polly-with-vba

I've picked the bones out of his code (i.e. the bits I need for hashing) & thankfully have ended up with the same hash result as the data on amazon's sample/test page

here are the key bits

Public Function HashTest()
Call getSignatureKey("AWS4wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY", "20120215", "us-east-1", "iam")
End Function

Private Function getSignatureKey(key As String, dateStamp As String, regionName As String, serviceName As String) As Byte()

    Dim kDate() As Byte, kRegion() As Byte, kService() As Byte, kSigning() As Byte
    kDate = hmacSha256(key, dateStamp)
    kRegion = hmacSha256(kDate, regionName)
    kService = hmacSha256(kRegion, serviceName)
    kSigning = hmacSha256(kService, "aws4_request")

    getSignatureKey = kSigning

    Debug.Print byte2hex(getSignatureKey)   '<< this is the key bit, it gives the hash result  f4780e2d9f65fa895f9c67b32ce1baf0b0d8a43505a000a1a9e090d414db404d ...which is what it should be   :-)

End Function
Public Function hmacSha256(key As Variant, stringToHash As Variant) As Byte()

    Dim ssc As Object
    Set ssc = CreateObject("System.Security.Cryptography.HMACSHA256")

    ssc.key = str2byte(key)
    hmacSha256 = ssc.ComputeHash_2(str2byte(stringToHash))

    Set ssc = Nothing

End Function
Public Function sha256(stringToHash As Variant) As Byte()

    Dim ssc As Object
    Set ssc = CreateObject("System.Security.Cryptography.SHA256Managed")

    sha256 = ssc.ComputeHash_2(str2byte(stringToHash))

    Set ssc = Nothing

End Function
Public Function str2byte(s As Variant) As Byte()

    If VarType(s) = vbArray + vbByte Then
        str2byte = s
    ElseIf VarType(s) = vbString Then
        str2byte = StrConv(s, vbFromUnicode)
    Else
        Exit Function
    End If

End Function
Public Function byte2hex(byteArray() As Byte) As String

    Dim i As Long
    For i = 0 To UBound(byteArray)
        byte2hex = byte2hex & Right(Hex(256 Or byteArray(i)), 2)
    Next
    byte2hex = LCase(byte2hex)

End Function

....this should get me moving along, until I come unstuck at the next point!

Many thanks for lending an ear & the offer of help (I've been on this a few nights - typical isn't it? i.e., as soon as posing the question, my mate Google solves it within a few hours!)

Hi. Congratulations! Good luck with the rest of your project.

 

[isladogs]

Hi again
It may be too late but the person I contacted isn't an AWF member so cannot reply. However their response was

The issue is they convert a key string encoded as utf8 to binary so the MAC can be calculated. But they return the binary MAC as a hex representation string.

They then go on to use that hex representation to calculate another MAC. But the problem is that their function is expecting binary data encoded as a utf8 encoded string.

They should separate the encodings to binary and binary to encoding functionality from the MAC calculation. Have the MAC calculation work on binary data. And call the encoding translations as necessary to convert between binary data and display encodings.

I hope that helps

 

[peskywinnets]

Hi again
It may be too late but the person I contacted isn't an AWF member so cannot reply. However their response was


I hope that helps

Thank your friend for me ..he's bang on the money

 

[isladogs]

Will do. Glad to act as intermediary