DevSecOps: Looking at Code Signing Holistically

oking at Code Signing Holistically

Code signing proves the identity of the source vendor of a software (only the private code signing key holder can create the signature) and verifies that the code has not been tampered with since being published. Code signing can be done in two phases, at the first once binary files (library/executables) are signed, and in the latter packages containing collections of such (e.g. RPM, docker, NuGet) are signed in order to provide means to validate the released packaged code.

Code Signing Will Grow in Importance and Complexity

Requirements for code signing will grow as single monolithic apps are replaced by multiple microservices developed by disparate teams, service velocity leads to more frequent releases, and DevOps processes will include code signing in varied stages of the lifecycle prior to release.

Challenges in Implementing Secure Code Signing

As code signing grows in both importance and complexity as a security best practice, there also exist challenges in implementing secure code signing solutions within organizations. There exist a number of code signing approaches, and each approach has its respective challenges.

Basic: Unmanaged, Unsecured (DIY 1)
Unmanaged & Unsecure Code Sign
  • Private keys are placed in signing servers or users’ endpoint devices – not in secure storage, creating a very real possibility of signing keys compromise
  • No centralized key management, no administrative control – everything must be done manually
  • No capability to enforce security policies consistently.
    HSM for Code Signing
    HSM for Code Signing

     

  • HSMs are difficult to manage, DevOps teams that are in charge typically do not possess the capacity to manage an HSM.
  • Using HSMs would require a very large number (due to the number of the build servers in an organization) and variety of clients (due to the need to support various platforms), making HSM usage a real nightmare of constantly resolving signer integration issues. This can be avoided by taking the “central signing server” approach, which creates other problems (latency and security issues with sending the files to be signed to the central signing server); the comparison of these approaches will be discussed below.
  • Key usage control is a security issue (the code signing keys are well-protected in the HSM, but such solutions doesn’t allow to control what is signed)
  • No ability for supporting ephemeral clients (e.g. signing from containers)
  • This is not a product working OOB, and requires implementing integration with each of the required signers / development infrastructures
  • Large organizations have difficulty synchronizing keys across different geographies
  • Not supporting distributed architecture
  • Very basic support for file types (mostly MS), cannot sign RPM or Mac
Code Signing with optional HSM
Code Signing (with optional HSM)

 

  • Current leading solutions have no integration with public CAs, making the certificate purchase and renewal cumbersome, and having to use an additional tool for certificates management
  • No built-in key usage control measures – this is especially important as we have seen a number of hacks (as in the case of ASUS, see more here) where the hackers succeeded to use the existing code signing solution to sign code that included malware
  • Requires managing HSMs, as it’s not an integral part of the code signing solution
  • Lacks proper client security
  • No integration with signing tools/build servers
  • Limited platforms (mostly MS)
    Code Signing SaaS
    Code Signing SaaS

     

  • If signing large files is required there is a need to keep on-prem instance (for performing the hash operation)
  • Lack of filetype support can make the service irrelevant, dependent on the specific list of file types each service provider supports
  • Integration only with one public CA (this is the case as the only SaaS vendors in the market currently are CAs)
  • The end customer cannot manifest control over the code signing keys as the keys are held by the service provider

Local and Remote Code Signing

One of the specific issues that we would like to delve into is a comparison between the local and remote signing options. When evaluating a code signing approach, an often overlooked but nonetheless important consideration is its ability to perform local and remote code signing. For example, the “Unmanaged & Unsecure” and “HSM for Code Signing” approach only allow for local code signing, whereas the SaaS approach only does remote (while also offering local due to performance issues). A dedicated end to end code signing solution can do both, depending on the implemented topology.

ParameterLocal Signing Central / Remote Signing
PerformanceThe hash is created locally, while the signature is created at the secure location of the code signing keys storage (i.e. in vHSM/HSM), distributing the overall required performance across all the building servers in the organization, across different times, no delay penalty. Transferring large files from the local build servers to the location of the centralized code signing system, can cause a substantial delay, hindering the release cycle. It also requires a centralized solution that can cope with the required performance, compared with the distributive nature of the local signing.
Integration with signers* and development environments If the signing is done locally there is a seamless integration between the signer and the security provider. IDE** such as Visual Studio that utilize local signing are extremely difficult to allow centralized code signing due to the need to perform an integration with the security stack of the machine that the client is installed on (i.e. JCA*** , MS KS**** )If the signing is performed remotely, there is a need to have all the relevant platforms (i.e. all types of Linux distributions, Windows servers versions, Mac, etc. that exist in the organization in a central location), and manually configure the security providers for each signer.

 

End-Customer Perspective

R&D and DevOps leaders are looking to automate, speed and simplify the code signing process. Code signing must be integrated within the DevSecOps processes and enhance it by improving operational efficiency. Different development teams might have different SDLC flows, in which the code signing solution must naturally fit, without causing disturbance the in the development process. A proper code signing solution should support the existing platforms and development environments, as well as respond quickly to changing market demands – support for additional signers, platforms, and development environments (microservices, containers).

Summary

It is essential for all the relevant stakeholders in the enterprise that are involved in code signing operations (such as CIO, VP R&D, CISO, Head of DevOps) to have a holistic overview of the code signing process – what is signed, on which platforms, what type of certificates are used (OV/EV), how the certificates are managed, how the code signing keys are protected, how the entire solution is managed and automated, who have permissions to do what, how the users are authenticated, etc., and come up with the most suitable course for the organization.


[*] Signers are dedicated SW that are used to create hash and sign it in specific software environments (i.e. Jarsigner that is used to sign java files).
[**] Integrated development environment
[***] Java Cryptography Architecture
[****] Key Storage Provider
George Wainblat

George Wainblat

George Wainblat joined Unbound in June 2017 as Director of Product Management. George brings a wealth of experience in leading multi-disciplinary product, engineering and business units at global hi-tech companies as well as startups.

Subscribe to BLOG

shares