Code signing proves the identity of the source vendor of a software (only the private code signing key holder can create the signature) and verifies that the code has not been tampered with since being published. Code signing can be done in two phases, at the first once binary files (library/executables) are signed, and in the latter packages containing collections of such (e.g. RPM, docker, NuGet) are signed in order to provide means to validate the released packaged code.
Code Signing Will Grow in Importance and Complexity
Requirements for code signing will grow as single monolithic apps are replaced by multiple microservices developed by disparate teams, service velocity leads to more frequent releases, and DevOps processes will include code signing in varied stages of the lifecycle prior to release.
Challenges in Implementing Secure Code Signing
As code signing grows in both importance and complexity as a security best practice, there also exist challenges in implementing secure code signing solutions within organizations. There exist a number of code signing approaches, and each approach has its respective challenges.
- Private keys are placed in signing servers or users’ endpoint devices – not in secure storage, creating a very real possibility of signing keys compromise
- No centralized key management, no administrative control – everything must be done manually
- No capability to enforce security policies consistently.
- HSM for Code Signing
- HSMs are difficult to manage, DevOps teams that are in charge typically do not possess the capacity to manage an HSM.
- Using HSMs would require a very large number (due to the number of the build servers in an organization) and variety of clients (due to the need to support various platforms), making HSM usage a real nightmare of constantly resolving signer integration issues. This can be avoided by taking the “central signing server” approach, which creates other problems (latency and security issues with sending the files to be signed to the central signing server); the comparison of these approaches will be discussed below.
- Key usage control is a security issue (the code signing keys are well-protected in the HSM, but such solutions doesn’t allow to control what is signed)
- No ability for supporting ephemeral clients (e.g. signing from containers)
- This is not a product working OOB, and requires implementing integration with each of the required signers / development infrastructures
- Large organizations have difficulty synchronizing keys across different geographies
- Not supporting distributed architecture
- Very basic support for file types (mostly MS), cannot sign RPM or Mac
Code Signing with HSM
- Current leading solutions have no integration with public CAs, making the certificate purchase and renewal cumbersome, and having to use an additional tool for certificates management
- No built-in key usage control measures – this is especially important as we have seen a number of hacks (as in the case of ASUS, see more here) where the hackers succeeded to use the existing code signing solution to sign code that included malware
- Requires managing HSMs, as it’s not an integral part of the code signing solution
- Lacks proper client security
- No integration with signing tools/build servers
- Limited platforms (mostly MS)
- If signing large files is required there is a need to keep on-prem instance (for performing the hash operation)
- Lack of filetype support can make the service irrelevant, dependent on the specific list of file types each service provider supports
- Integration only with one public CA (this is the case as the only SaaS vendors in the market currently are CAs)
- The end customer cannot manifest control over the code signing keys as the keys are held by the service provider
Local and Remote Code Signing
One of the specific issues that we would like to delve into is a comparison between the local and remote signing options. When evaluating a code signing approach, an often overlooked but nonetheless important consideration is its ability to perform local and remote code signing. For example, the “Unmanaged & Unsecure” and “HSM for Code Signing” approach only allow for local code signing, whereas the SaaS approach only does remote (while also offering local due to performance issues). A dedicated end to end code signing solution can do both, depending on the implemented topology.
R&D and DevOps leaders are looking to automate, speed and simplify the code signing process. Code signing must be integrated within the DevSecOps processes and enhance it by improving operational efficiency. Different development teams might have different SDLC flows, in which the code signing solution must naturally fit, without causing disturbance the in the development process. A proper code signing solution should support the existing platforms and development environments, as well as respond quickly to changing market demands – support for additional signers, platforms, and development environments (microservices, containers).
It is essential for all the relevant stakeholders in the enterprise that are involved in code signing operations (such as CIO, VP R&D, CISO, Head of DevOps) to have a holistic overview of the code signing process – what is signed, on which platforms, what type of certificates are used (OV/EV), how the certificates are managed, how the code signing keys are protected, how the entire solution is managed and automated, who have permissions to do what, how the users are authenticated, etc., and come up with the most suitable course for the organization.